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TERRAIN  ANALYSIS  FOR  HUMAN-ROBOT  INTERACTION  (TAH-RI):  ENABLING 
TERRAIN  UNDERSTANDING  TO  IMPROVE  TACTICAL  BEHAVIOR 

EXECUTIVE  SUMMARY 


In  this  feasibility  study,  we  investigated  implications  of  the  fact  that  military  decision-making  for 
ground  forces  is  driven  by  tactical  constraints  and  opportunities  based,  in  large  part,  on  terrain. 
Future  Force  Warrior  (FFW)  and  Future  Combat  Systems  (FCS)  initiatives  are  developing 
advanced  functional  capabilities  to  aid  Soldiers  in  operations  to  control  and  hold  ground. 

Adding  robotic  vehicles,  sensors,  and  weapons  creates  a  planning  and  coordination  challenge  for 
commanders,  and  highlights  the  need  for  autonomous  robotic  systems  that  effectively 
“understand”  the  tactical  import  of  terrain  and  integrate  that  understanding  into  their  situation 
awareness  and  behavior-generation  processes.  Means  of  increasing  readiness  of  Soldiers  (e.g., 
through  training  and  performance  support)  to  properly  integrate  terrain  understanding  into  their 
own  battlefield  decision-making  processes  are  needed,  as  are  means  of  enabling  more  autonomy 
in  robots  through  terrain  understanding  for  tactical  behavior  generation. 


Procedure 

In  this  study,  we  assessed  terrain  analysis  capabilities  of  existing  digital  tools  to  help  determine 
if,  and  how,  output  from  these  tools  could  be  utilized  as  inputs  for  training  humans,  aiding 
humans  during  operations,  and  informing  intelligent  agents  and  autonomous  robots  of  tactically 
important  terrain  features.  Analyzed  means  of  integrating  terrain  analysis  data,  intelligence  data, 
and  the  Common  Relevant  Operating  Picture  (CROP)  to  aid  path  planning  and  execution.  Also 
analyzed  means  of  representing  and  reasoning  about  commander’s  intent  and  mixed-initiative 
battlefield  communications  among  combinations  of  humans  and  robots.  Completed  initial  design 
of  composable  software  architecture  enabling  rapid  integration  into  existing  digital  C2  systems, 
Intelligent  Tutoring  Systems,  and  advanced  robots. 


Findings 

Existing  digital  tools  for  aiding  and  training  humans  are  inadequate  and  must  be  improved. 
Technology  to  better  enable  humans,  and  especially  robots,  to  effectively  understand  terrain  and 
its  tactical  import  is  needed.  Highlighting  areas  of  terrain  onscreen  can  help  a  human  Soldier, 
but  an  autonomous  robot  or  an  Intelligent  Tutoring  System  (ITS)  needs  a  means  to  efficiently 
represent  and  reason  about  what  is  being  highlighted  for  a  human  to  see.  A  novel  Terrain 
Analysis  for  Human-Robot  Interaction  (TAH-RI)  software  sub-assembly  was  designed  to  solve 
this  problem,  based  on  state-of-the-art  terrain  analysis  algorithms,  intelligent  agent  software,  and 
other  technologies  developed  to  aid  FCS  and  FFW.  Importantly,  TAH-RI  incorporates  terrain 
analysis  tools  that  are  the  next  generation  of  CHI  Systems’  software  components  fielded  in  the 
Marine  Corps’  Command  and  Control  Personal  Computer  (C2PC). 


Utilization  of  Findings 

Build  TAH-RI  sub-assembly  and  demonstrate  its  utility  for  enhancing  existing  tools. 
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Introduction 


Terrain  has  a  big  impact  on  how  battlefield  situations  unfold  primarily  because  of  its 
effects  on  intervisibility,  mobility,  and  restriction  of  fields  of  fire.  As  armed  forces  of  the 
information  age  come  within  each  other’s  sensor  coverage,  information  about  them  is  rapidly 
conveyed  to  their  opponents.  Terrain  can  play  a  significant  role  in  determining  their  effective 
sensor  coverage  area.  Terrain  and  ground  cover  also  can  play  a  significant  role  in  constraining, 
to  varying  degrees,  mobility  of  vehicles  and  dismounted  infantry  and  scouts.  Terrain  also  can 
constrain  engagability  by  restricting  possible  fields  of  fire  for  Line  of  Sight  (LOS),  Extended 
LOS  (ELOS),  Non-LOS  (NLOS),  and  Beyond  LOS  (BLOS)  weapons  systems.  Thus, 
understanding  the  terrain  of  the  battlespace,  and  its  tactical  import  for  military  operations  is 
essential  if  a  force  is  to  succeed  in  its  missions.  This  is  why  understanding  the  terrain  comes 
immediately  after  the  mission  and  the  enemy  in  the  US  Army  Training  and  Doctrine  Command 
(TRADOC)  guidance  to  commanders  to  approach  battlefield  decision-making  by  considering  the 
Mission,  Enemy,  Terrain,  Troops,  and  Time  (METT-T).  Terrain  imposes  constraints  and  opens 
opportunities  for  the  creative  use  of  Battlefield  Operating  Systems  (BOS)  and  the  capabilities 
and  limitations  of  available  troops,  vehicles,  systems,  and  materiel. 

Understanding  the  impact  of  terrain  on  plans  and  actions  can  be  hard  for  people.  This  is 
true  in  both  military  and  civilian  endeavors,  although  the  training  that  comes  with  military 
service  certainly  can  help  in  this  regard.  Integrating  terrain  understanding  with  plans  to  achieve 
both  shared  and  individual  goals  can  lead  to  less  than  optimal  results  (e.g.,  as  with  buddies  out 
hunting  who  get  lost  or  separated),  as  illustrated  in  the  humorous  writings  of  Patrick  McManus: 

“Every  hunter  knows  what  a  rendezvous  is.  That’s  where  one  hunter  says  to 
another,  ‘Al,  you  take  that  side  of  the  draw  and  I’ll  take  this  one  and  we  ’ll  meet  in 
twenty  minutes  at  the  top  of  the  hill.  ’  The  next  time  they  see  each  other  is  at  a 
PTA  meeting  five  years  later  in  Pocatello.  That ’s  a  rendezvous. 


It  is  simply  against  the  basic  nature  of  hunters  to  arrive  at  a  designated  point  at  a 
designated  time.  If  one  of  my  hunting  pals  said,  ‘I’ll  meet  you  on  the  other  side  of 
this  tree  in  ten  seconds, '  one  of  us  would  be  an  hour  late.  And  have  the  wrong 
tree  besides.  ”  [McManus,  1981,  p.  1 74] 

In  most  cases,  however,  these  problems  can  have  much  more  serious  results.  On  the  battlefield, 
the  results  can  be  tragic  and  of  historic  proportions,  as  in  the  Civil  War  at  the  battle  of 
Gettysburg  where  not  every  commander  present  on  those  fateful  days  truly  understood  the 
potential  impact  of  the  terrain  of  that  battlefield  on  their  plans  and  intended  actions  (cf.,  Shaara, 
1975).  Better  means  of  training  people  to  understand  the  tactical  import  of  terrain  are  needed. 
Better  means  of  performance  support  for  people  with  a  lot  more  than  a  deer  on  their  mind  are 
needed  to  help  properly  factor  terrain  into  on-going  activities  and  plans.  Similarly,  means  of 
enabling  the  Army’s  envisioned  robotic  systems  to  understand  terrain  and  its  tactical  import 
(e.g.,  to  be  able  to  represent  and  reason  about  METT-T  in  an  efficient  yet  effective  way)  is 
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essential  to  their  future  battlefield  utility.  If  properly  understanding  terrain  import  is  hard  for 
people,  imagine  the  difficulties  encountered  by  autonomous  robots  in  performing  this  feat! 

Purpose  and  Organization  of  the  Report 

This  technical  report  is  the  Final  Report  for  the  Terrain  Analysis  for  Human-Robot 
Interaction  (TAH-RI)  Phase  I  STTR  effort  for  contract  number  W74V8H-04-P-0482,  CDRL 
0002  and  covers  the  period  of  performance  for  Phase  I,  from  7  August  2004  through  7  February 
2005.  The  research  reported  here  focused  on  designing  and  assessing  feasibility  of  a  solution  to 
the  terrain  understanding  problem,  called  Terrain  Analysis  for  Human-Robot  Interaction  (TAH- 
RI),  a  novel  software  sub-assembly  which  can  be  integrated  into  a  variety  of  systems: 

•  to  enable  more  effective  solutions  for  Soldiers  through 
o  training  systems  and  Intelligent  Tutoring  Systems, 
o  decision  aiding  subsystems,  and 

o  performance  support  systems 

•  to  enhance  autonomy  and  utility  of  tactical  robotic  systems  by  enabling 
o  terrain  analysis, 

o  more  effective  representations  of  analytic  results, 
o  integration  of  those  results  with  CROP  and  Intelligence,  and 
o  much  more  efficient  human-robot  interaction  is  also  needed 

■  with  unit  commanders 

■  with  command  post  staff  or  other  Soldiers  tasked  with  using  robots 

■  with  other  Soldiers  fighting  near  robots 

■  with  (apparent)  non-combatants  near  robots. 

Details  of  the  work  completed  in  Phase  I,  and  the  insights  gained  into  feasibility  of  achieving  the 
stated  goals  for  TAH-RI  in  Phase  II  development  and  evaluation,  are  described  in  this  report. 


Background 

As  the  transformation  of  the  Armed  Forces  of  the  United  States  unfolds,  the  importance 
of  enabling  our  Soldiers  to  shape  the  battle  before  coming  in  contact  with  the  enemy  is 
significantly  increasing.  This  is  especially  true  for  the  smallest  units  (e.g.,  traditional  platoons 
and  squads,  teams  and  cells,  scouts  and  individual  Future  Force  Warriors)  which  must  be  able  to 
achieve  enhanced  lethality,  survivability,  tactical  awareness,  mobility,  sustainability,  training  and 
readiness  to  overcome  both  traditional  and  asymmetric  threats  as  the  information  age  shapes  the 
battlespace  of  the  21st  century.  Meeting  these  challenges  is  becoming  more  viable  as  innovative 
new  capabilities  for  the  Future  Force  Warrior  (FFW)  and  Future  Combat  System  of  Systems 
(FCS)  are  rolled  out.  However,  significant  challenges  remain. 

One  of  the  remaining  critical  challenges  to  ensuring  realization  of  this  potential  for 
shaping  the  battle  well  before  contact  is  the  integration  of  terrain  understanding  with  a  Common 
Relevant  Operations  Picture  (CROP)  enabling  Soldiers  to  understand  the  battlefield  situation  and 
the  tactical  importance  of  the  terrain  across  which  the  battle  is  unfolding,  especially  on  complex 
or  urban  terrain.  Building  and  maintaining  situation  awareness  of  the  relevant  areas  and  features 
of  the  battlespace  is  vital  to  effective  and  efficient  employment  of  the  Future  Force’s  technical 
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advantages  over  conventional  and  asymmetric  forces.  Many  Soldiers  develop  significant 
expertise  in  performing  this  integration  on-the-move  with  available  information  -  but  not  all 
Soldiers  develop  this  level  of  competence.  Thus,  intelligent  software-driven  Electronic 
Performance  Support  Systems  (EPSSs),  or  Decision  Aids  (DAs),  and  Intelligent  Tutoring 
Systems  (ITSs)  are  needed  to  ensure  that  all  of  the  human  warfighters  in  these  units  can 
effectively  and  efficiently  employ  all  of  these  advanced  capabilities  to  achieve  victory  on  the 
future  battlefield.  Of  special  interest  is  the  envisioned  integration  of  a  variety  of  robotic 
vehicles,  sensors,  weapons  systems,  and  other  kinds  of  robotic  systems  intended  to  aid  lower- 
echelon  commanders  “at  the  pointy  end  of  the  stick”  to  master  the  battlespace,  even  when  the 
terrain  turns  from  open  and  rolling  to  complex  and  urban. 

If  the  problem  of  understanding  the  unfolding  situation  is  critical  and  difficult  for  human 
combatants,  it  is  even  more  so  for  the  envisioned  robotic  combatants  and  support  systems. 
Robotic  systems  will  be  participating  in  armed  conflicts  in  the  coming  decades,  but  will  they 
enhance  or  degrade  the  abilities  of  the  human  warfighters  beside  them  and  of  their  commanders? 
While  holding  out  the  promise  of  significantly  enhancing  a  commander’s  ability  to  build  and 
maintain  an  accurate  CROP,  robotic  sensor  systems  also  threaten  to  sap  a  commanders’  attention 
given  current  robotic  technologies  inability  to  achieve  autonomy.  Rather  than  dramatically 
increasing  a  human  commander’s  span  of  control  with  not  only  more  “troops”  but  more  “dumb” 
troops,  robotic  systems  need  to  be  “smart”  enough,  robust  enough,  and  autonomous  enough  to 
minimize  impact  on  the  forces  with  which  they  are  fielded.  To  be  effectively  (and  efficiently) 
autonomous,  these  robotic  systems  must  interact  with  their  human  commanders  and  the  major 
Battlefield  Operating  Systems  (BOS)  in  which  they  play  vital  roles,  and  must  have  access  to 
relevant  data,  information,  analysis  tools,  and  results  of  analyses.  Whether  engaged  in  combat, 
combat  support,  or  combat  service  support,  each  of  these  robotic  systems  must  “understand”  and 
act  upon  their  commander’s  intent,  their  mission  and  specific  objectives,  and  the  battlefield 
environment.  Obviously,  an  armed  robotic  scout  will  need  both  significant  autonomy  to  move 
about  the  battlespace  to  achieve  needed  sensor  (and  possibly  weapons)  coverage  while  still 
requiring  significant  human  oversight  and  control  of  the  robot’s  weapon(s). 

Understanding  the  battlefield  environment  also  involves  more  than  just  “knowing”  your 
location,  more  than  just  finding  a  navigable  route  over  the  relevant  terrain  to  reach  specified 
waypoints.  Rather,  it  involves  some  level  of  “knowing”,  integrating,  maintaining,  and 
effectively  using  their  whole  CROP,  including  relevant  intelligence  information  and  assessments, 
relevant  operations  information  and  plans,  and  the  tactical  importance  of  the  terrain  in  their  area 
of  operations  and  area  of  interest.  Autonomous  robotic  systems  need  to  build  and  maintain,  and 
then  act  upon,  their  own  awareness  of  the  battlefield  situation  (recent  past,  current,  and  projected 
future).  These  systems  need  to  be  able  to  at  least  read  and  use  the  products  of  the  Intelligence 
Preparation  of  the  Battlefield  (IPB)  incorporated  within  the  mission  and  orders  they’ve  received, 
and  integrate  their  own  local  knowledge  with  this  larger  picture.  Some  of  these  systems  will 
even  be  essential  participants  in  the  production  and  on-going  maintenance/revision  of  the  data 
and  information  contained  in  those  IPB  products.  These  needs  also  apply  to  the  intelligent 
software  required  to  enable  effective  intelligent  Decision  Aiding  (DA)  and  Intelligent  Tutoring 
Systems  (ITSs)  to  aid  and  train  the  human  warfighters  in  building,  maintaining,  and  using  their 
own  understanding  of  the  CROP  and  battlespace  terrain. 

Another  important  aspect  of  this  set  of  problems  is  more  directly  in  the  area  of  human- 
robot  interaction.  Because  of  the  significant  differences  between  human  beings  vs.  robotic 
systems  as  subordinates,  humans  controlling  robotic  systems  can  be  surprised  by  the  unexpected 
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kinds  of  errors,  mistakes,  misunderstandings,  and  failures  that  can  occur.  Thus,  in  addition  to 
formulating  and  following  this  guidance,  robotic  systems  also  must  be  able  to  explain  what  they 
are  doing  or  intend  to  do,  when  they  intend  to  do  it,  and  why  they  are  doing  it  at  this  time  and 
place  given  the  current  circumstances  and  its  understanding  of  those  circumstances. 

Importance  of  Autonomy  for  Robotic  Systems 

For  robotic  systems,  a  key  challenge  is  acquiring,  understanding,  and  integrating 
disparate  elements  of  information  describing  the  total  battlespace,  including  terrain  effects  on 
operations  planning  and  execution.  This  includes  integration  of  physical  environment  and 
topography  information  with  the  current  and  anticipated  distribution  of  forces  across  the 
battlespace  available  through  operations  status  reports  and  plans  (primarily  about  own  forces) 
and  intelligence  analyses  and  reports  (primarily  about  opposing  forces).  For  human  warfighters, 
viewing  a  topographic  map  overlaid  with  unit  symbols,  targets,  maneuver  graphics,  and  other 
assorted  information  renderings  can  sometimes  be  sufficient  to  enable  cooperative  work  to 
achieve  a  given  unit’s  mission.  For  robots,  it’s  a  little  bit  harder. 

As  FCS  and  FFW  weapon  systems  approach  fielding,  including  robotic  missile,  gun, 
sensor,  reconnaissance,  logistics,  and  search  and  rescue  systems,  the  human  Soldiers  who  must 
command  and  control  these  robotic  systems  will  be  faced  with  an  extremely  difficult  span  of 
control  problem.  Other  human  Soldiers  who  depend  upon  these  autonomous  robotic  systems  to 
fulfill  various  combat,  combat  support,  and  combat  service  support  functions  will  also  be  faced 
with  a  significant  situation  awareness  and  human-computer  interaction  problem,  because  these 
systems  will  often  not  behave  in  human-like  ways  and  will  have  very  different  capabilities  and 
limitations  than  their  human  counterparts. 

As  an  example,  the  span  of  control  problem  for  human  warfighters  was  amply 
demonstrated  in  the  May  2001  Future  Combat  Command  and  Control  (FCC2)  experiment  at  the 
Mounted  Maneuver  Battlespace  Lab  (MMBL).  During  this  experiment,  in  which  CHI  Systems 
participated,  a  typical  FCS-equipped  Cell  (i.e.,  platoon)  was  composed  of  a  single  command  and 
control  vehicle  with  three  personnel,  one  of  which  was  a  driver,  two  robotic  reconnaissance 
vehicles,  two  robotic  multi-role  guns,  a  robotic  mortar,  robotic  missile  launcher  vehicle,  and  a 
robotic  uninhabited  aerial  reconnaissance  platform  (UAV).  Additionally,  the  robotic 
reconnaissance  vehicles  could  deploy  up  to  40  uninhabited  ground  sensors  (UGS).  Typically,  a 
cell  leader  was  required  to  command  and  control  up  to  45  robotic  vehicles  and  sensors,  all  of 
which  required  individual  attention  in  order  to  cause  the  systems  to  perform  various  functions  in 
a  timely  fashion  to  achieve  overall  unit  objectives.  CHI  Systems’  observation  of  the  FCS  cell 
leaders  and  their  assistants  revealed  that  they  were  essentially  overwhelmed  by  the  need  to 
control  the  numerous  robotic  systems  at  their  command.  This  occurred  when  two  personnel 
were  attempting  to  control  the  robotic  systems;  however,  during  continuous  operations,  when 
only  one  human  controller  was  available,  even  this  limited  ability  to  control  the  robotic  systems 
was  degraded  to  the  point  where  control  was  completely  ineffective.  During  the  course  of  a  prior 
project,  one  of  CHI  Systems’  personnel  attempted  to  control  the  various  robotic  vehicles  of  an 
FCS  cell  on  numerous  occasions,  with  similar  results.  It  was  literally  impossible  to  maintain 
awareness  of  all  the  robotic  systems. 

Another  example  of  the  need  for  robust  autonomy  in  terrain  interpretation  in  robotic 
systems  occurred  during  the  recent  DARPA  Grand  Challenge  competition  in  March  2004.  As 
reported  at  the  outbriefing  at  the  DARPA/DSRC  Workshop  on  Hardware  Fabric  on  Intelligent 
Machines  immediately  following  the  competition,  one  of  the  larger  competing  robotic  vehicles 
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kicked  up  significant  debris  and  boulders  onto  the  road.  An  entry  that  followed  later  relied 
primarily  on  the  previous  model  of  the  road  and  surrounding  terrain.  As  a  result,  the  vehicle 
expected  an  uncluttered  surface  and  attempted  to  navigate  at  a  higher  speed  than  was  safe  for  the 
rapid  reactive  turns  required  to  avoid  the  rocks.  Because  it  was  not  programmed  to  understand 
the  actual  terrain,  the  vehicle  could  not  adapt  and  quickly  failed.  This  example  illustrates  that  a 
realistic  robotic  terrain  interpretation  system  must  rely  on  both  top-down  terrain  modeling  (a 
priori  maps)  and  bottom-up  (situated  learning)  to  be  aware  of  the  terrain  and  make  the 
appropriate  decisions. 

Importance  of  Terrain  for  Military  Operations 

Terrain  can  significantly  affect  application  of 
each  of  the  9  principals  of  war,  and  understanding  of 
terrain,  or  lack  thereof,  may  be  the  difference  between 
victory  and  defeat.  The  potential  impact  on  each 
principle  is  summarized  below: 

•  Objective:  A  clear  objective  is  required  in 
order  to  achieve  the  mission;  lack  of 
understanding  of  the  terrain  may  mean  the 
desired  objective  is  unattainable. 

•  Offensive:  Offensive  action  wins  battles; 
lack  of  understanding  of  terrain  may  inhibit 
the  ability  to  remain  on  the  offensive  as 
units  bog  down. 

•  Mass:  Massing  combat  power  at  the  critical  place  and  time  is  key;  lack  of 
understanding  of  terrain  may  lead  to  inability  to  mass. 

•  Maneuver:  Allows  forces  to  mass  and  disperse  as  needed  to  keep  the  initiative;  lack 
of  understanding  of  terrain  slows  maneuver  and  risks  loss  of  the  initiative. 

•  Economy  of  Force:  Risk  must  be  accepted  in  some  areas  in  order  to  mass  combat 
power;  Terrain  is  one  of  the  key  factors  in  economy  of  force  operations  and  lack  of 
terrain  understanding  leads  to  unacceptable  risk. 

•  Unity  of  Command:  A  single  commander  directs  the  force  toward  the  common 
objective;  failure  to  understand  terrain  may  not  allow  the  commander  to  position  so  as 
to  achieve  unity. 

•  Security:  Security  protects  combat  power;  not  understanding  the  terrain  degrades 
security  in  terms  of  observation,  fields  of  fire,  etc. 

•  Surprise:  Surprise  is  a  combat  multiplier;  failure  to  understand  terrain  may  lead  to 
premature  discovery  of  forces  attempting  surprise. 

•  Simplicity:  Plans  and  orders  must  be  simple  and  direct;  failure  to  understand  terrain 
may  cause  seemingly  simple  plans  to  become  complicated  during  execution. 

How  are  Terrain  Analysis  Tools  Used  Now? 

•  Intelligence  Preparation  of  the  Battlefield  (IPB). 

o  Analysis  of  the  terrain  and  terrain  effects  to  support  the  generation  and 
wargaming  of  friendly  and  enemy  courses  of  action. 


The  Principles  of  War 

•  Objective 

•  Offensive 

•  Mass 

•  Economy  of  force 

•  Maneuver 

•  Unity  of  command 

•  Security 

•  Surprise 

•  Simplicity 
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Situational  Templates  apply  terrain  effects  to  predict  likely  enemy  locations. 
Mobility  Corridors  to  identify  Avenues  of  Approach. 

Mobility  and  Combined  Obstacle  Overlays 

Foundation  for  IPB  Event  and  Decision  Support  templates  in  COA  wargaming. 


IPB  wizard  built  by  CHI  Systems  currently  fielded  with  USMC  in  C2PC 
application,  as  shown  in  Figure  1 ,  below.  IPB  Wizard  uses  our  terrain  component 
to  walk  user  through  IPB  process  and  provides  html-based  document  that  is  used 
as  part  of  intelligence  annex  to  operations  order. 
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Figure  1.  IPB  Wizard  in  USMC’s  C2PC 


Route  Planning  (fastest  route,  covered  and  concealed  routes) 

Time-based  Mobility  Range  Rings 

Identification  of  Restricted,  Severely  Restricted,  Unrestricted  mobility  areas 

Identification  of  mobility  corridors  for  different  tactical  echelons 

Line  of  Sight  for  optical  sensors  and  weapon  systems 

Aerial  Line  of  Sight 

Elevation  profiles 

Elevation  contour  maps 

Sensor  location  planning 

UAV  flight  route  planning 

Flight  path  planning  and  evaluation  for  Non-Line-of-Sight  Launch  System  (NLOS- 
LS)  Precision  Attack  Missile. 

•  Terrain  Macros 

o  Likely  helicopter  landing  zones 
o  Likely  artillery  firing  positions 

o  Likely  choke  points  (Named  and  Targeted  Areas  of  Interest  (NAI  &  TAI) 
o  Likely  and  known  bridging  sites 
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Examples  of  how  people  use  terrain  analyses  with  current  state-of-the-art  terrain  analysis 
algorithms  are  shown  in  Figure  2,  below.  Furthermore,  terrain  (and  its  effects)  directly  affect 
almost  every  step  in  the  military  decision-making  process  (illustrated  in  Figure  3). 


Figure  2.  Examples  of  Terrain  Analyses  for  a  Variety  of  Uses  (Human-Readable) 


Commander's  Actions 


SH  Actions 


Figure  3.  Importance  of  Terrain  in  the  Military  Decision-Making  Process 


Project  Overview 

The  overall  STTR  program  objective  for  Phase  I  (i.e.,  feasibility  assessment  and 
conceptual  design)  and  Phase  II  (i.e.,  detailed  design  and  development)  is  to  enable  human 
decision  makers,  intelligent  software  agent  intermediaries  (i.e.,  disembodied  robots),  and 
autonomous  robotic  systems  to  effectively  and  efficiently  use  terrain  analysis  results  to  enhance 
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battlefield  decision-making.  Specifically,  for  this  6-month  Phase  I  effort,  our  objective  is  stated 
below. 

Military  decision-making  for  ground  forces  is  driven  in  large  part  by  tactical  constraints 
and  opportunities  based  on  terrain.  The  US  Army’s  Future  Force  Warrior  (FFW)  and  Future 
Combat  Systems  (FCS)  programs  are  developing  advanced  functional  capabilities  to  greatly 
increase  effective  combat  power  of  our  warriors  on  land.  However,  despite  significant 
technological  advances,  Soldiers  still  have  to  take,  hold,  and  defend  tactically  significant  terrain. 
Adding  robotic  vehicles,  sensors,  and  weapons  systems  helps  in  some  ways,  but  also  can  create  a 
planning  and  coordination  challenge  for  the  human  commander.  The  addition  of  these  kinds  of 
systems  also  means  that  analyses  results  needed  for  effective  decision-making  must  be  in  a  form 
useful  for  enabling  ‘understanding’  by  robots  as  well  as  humans.  Coordinating  all  these  assets  is 
not  easy,  even  at  the  FCS  Cell  level.  Terrain  Analysis  for  Human-Robot  Interaction  (TAH-RI)  is 
intended  to  ensure  that  any  human  or  robot  who  needs  to  understand  the  integrated  tactical 
importance  of  terrain,  intelligence,  and  CROP  (Common  Relevant  Operating  Picture)  can 
understand  it,  along  with  the  commander’s  intent.  The  overall  goal  is  to  bridge  the  remaining 
critical  gaps  between  current  state-of-the-art  tools  in  terrain  analysis/reasoning  and  the 
operational  needs  of  the  warfighter  in  using  these  tools  for  command  and  control  of  forces  that 
include  varying  combinations  of  robotic  and  non-robotic  force  elements.  Based  on  a  review  and 
assessment  of  technology  status  and  mission  and  task  requirements,  we  will  design  and 
demonstrate  a  TAH-RI  system  solution  that  will  address  all  of  the  relevant  implementation  and 
performance  issues. 


Phase  I  Work  Summary 

The  specific  tasks  performed  in  Phase  I  are  summarized  below. 


Task  1 :  Assess  Terrain  Analysis  Capabilities  and  Unmet  Needs 

Assess  terrain  analysis  capabilities  of  existing  digital  tools  and  determine  if/how 
output  from  these  tools  can  be  utilized  as  inputs  for  training  humans,  aiding 
humans  during  operations,  and  informing  intelligent  agents  and  autonomous 
robots. 

Task  2:  Determine  What  Tool  Design  Changes  Needed 

Determine  what  design  changes  will  be  needed  in  selected  application  systems  to 
enable  integration  of  TAH-RI  as  a  sub-assembly  suitable  for  guiding  humans  and 
robots  in  understanding  and  utilizing  the  tactical  significance  of  specific  terrain. 

Task  3:  Investigate  Use  of  Commander’s  Mission  Intent 

Investigate  capture  of  commander’s  mission  intent  for  maneuver  into  computer 
readable- format,  so  that  it  can  be  used  in  the  TAH-RI  system.  This  will  include 
the  development  of  a  graphical  user  interface  to  help  enter  mission  details. 

CHI  Systems  has  significant  experience  with  this  problem  and  with  a  variety  of 
partial  solutions  that  have  been  attempted  in  the  past.  Truly  solving  this  problem 
requires  a  mixture  of  old  and  new  approaches.  Requirements  of  and  design  for 
this  GUI  are  being  refined  and  documented. 

Task  4:  Assess  How  to  Integrate  CROP  with  Robots 
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Assess  how  to  integrate  terrain  analysis  data,  intelligence  data,  and  the  operating 
picture  to  assist  path  planning  and  execution. 

Task  5:  Determine  What  Mixed-Initiative  Communications  are  Required 

Determine  what  mixed-initiative  communications  are  required  to  guide  route 

planning  and  execution.  Communications  could  be  between  human  and  intelligent 
agent,  human  and  robot,  or  intelligent  agent  and  robot.  How  to  communicate 
priorities  and  constraints  will  be  determined. 

Task  6:  Assess  Requirements  for  Real-Time  User-Updatable  Database(s) 

Assess  the  requirements  for  a  real-time  user-updatable  database(s)  of  terrain 
information  and  analysis  results. 

Task  7:  Complete  the  TAH-RI  Architecture 

Complete  the  TAH-RI  architecture,  combining  reusable  component  software, 

adaptive  cognitive  agents,  and  state-of-the-art  robotic  protocols.  Goal  is  to  create 
an  architecture  that  can  be  integrated  with  a  host  application  (such  as  an  existing 
digital  command  and  control  system). 

Task  8:  Document  Progress  and  Report  Results 

Document  progress  and  report  results  through  monthly  status  reports  and  through 
final  report  documenting  Phase  I  results  and  their  import  for  assessing  feasibility 
of  Phase  II/III  development  and  fielding  of  TAH-RI. 

The  results  achieved  in  performance  of  the  Phase  I  work  are  discussed  below,  by  task. 

Task  1  Results  :  Assess  Terrain  Analysis  Capabilities  and  Unmet  Needs. 

Our  assessment  of  key  existing  systems’  capabilities  resulted  in  identification  of  a 
number  of  unmet  needs.  We  discuss  these  results  below,  focusing  especially  on  the  key  issues  of 
terrain-sensitive  path  planning  and  the  broader  issues  of  navigation  and  means  of  enabling 
robotic  systems  to  integrate  and  understand  the  spatial  representation  of  threats,  terrain,  own 
forces,  and  some  of  the  many  other  factors  necessary  to  fully  understand  METT-T.  Many  of 
these  latter  issues  are  discussed  in  more  detail  further  below  with  results  of  Tasks  2  and  4. 

Existing  Digital  C2  Systems  and  Simulation  Environments. 

The  principal  digital  command  and  control  systems  for  land  warfare  (e.g.,  at  Echelons 
Brigade  and  Below)  currently  available  are  the  Army’s  Force  XXI  Battle  Command  Brigade  and 
Below  (FBCB2)  and  the  Marine  Corps’  Command  and  Control  Personal  Computer  (C2PC). 

Each  of  these  systems  provide  some  level  of  terrain  analysis  capability  to  its  users.  These 
capabilities,  and  those  of  the  ONESAF  Test  Bed  (OTB)  simulation  environment,  were  analyzed 
to  help  determine  what  terrain  analysis  needs  remain  unmet  by  available  digital  tools. 

During  Phase  I,  the  CHI  team  analyzed  existing  digital  tools’  (e.g.,  FBCB2,  C2PC,  OTB) 
terrain  analysis  capabilities,  using  available  information.  These  digital  tools  were  assessed  in 
terms  of  their  terrain  analysis  capabilities’  utility  for  training  humans,  aiding  humans  during  on¬ 
going  operations,  and  informing  intelligent  agents  and  autonomous  robots  regarding  terrain 
implications  for  current  and  anticipated  possible  courses  of  action  by  friend  and  foe.  Work  on 
this  task  involved  using  CHI  Systems’  experience  with  developing  software  components  for  the 
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USMC’s  C2PC,  and  our  experience  in  working  with  OTB-SAF  and  in  developing  a  variety  of 
training  and  intelligent  tutoring  systems.  Additionally,  our  in-house  expertise  on  terrain  analysis 
and  its  use  in  battlefield  planning  and  command  and  control  (C2)  was  used  to  identify  unmet 
needs.  Dr.  Robin  Murphy’s  robotics  expertise  significantly  enhanced  our  team’s  investigation  of 
the  issues  of  integrating  terrain  analysis  capabilities  with  existing  and  future  robotic  systems. 

Terrain-Sensitive  Path  Planning 

Terrain-sensitive  path  planning  is  an  emerging  area  of  concern  within  the  general  domain 
of  path  planning.  Path  planning  is  just  one  of  four  major  functions  necessary  for  robot 
navigation.  While  this  project  focuses  on  metric  path  planning,  it  is  helpful  to  have  an  overview 
of  navigation  and  path  planning  in  general  before  discussing  the  state  of  the  art  in  terrain- 
sensitive  path  planning  and  path  planning  for  teams  of  mobile  robots. 

Additionally,  our  team  completed  selection  of  algorithms  to  be  used  in  TAH-RI,  and  even 
implementation  and  integration  of  one  of  those  algorithms:  TRULLA.  This  work  is  informed  by 
our  review  of  literature  of  algorithms  for  real-time  terrain  interpretation  from  sensor  data,  robot 
systems  which  use  terrain  interpretation  from  maps  to  plan  and  navigate,  and  methods  of 
comparing  real-time  terrain  interpretation  with  predicted  terrain.  Toward  this  end,  our  team 
refined  a  set  of  preliminary  traversability  map  file  format  requirements  (i.e.,  the  format  for 
sharing  the  data  needed  by  wavefront  propagation  style  terrain  analysis  algorithms  like 
TRULLA).  Implementation  of  TRULLA  on  the  physical  robot  selected  for  Phase  I  development 
completed  using  the  latest  version  of  this  traversability  map  file  format.  An  illustration  showing 
the  resulting  field  produced  by  the  TRULLA  algorithm  is  illustrated  in  Figure  4,  below. 

Navigation  Overview 

Following  (Murphy,  2000),  robot  navigation  can  be  divided  into  four  questions,  each 
producing  a  distinct  set  of  algorithms:  where  am  I  going?  (mission  planning);  what’s  the  best 
way  there ?  (path  planning);  where  have  I  been?  (mapping  or  exploration);  and  where  am  I? 
(localization).  Note  that  tactical  navigation,  “ooops,  better  avoid  this  obstacle,”  is  considered  an 
execution  control  issue,  not  a  navigational  issue.  Path  planning  is  distinct  from  the  other  three 
functions.  Mission  and  path  planning  can  be  considered  knowledge-based  activities,  while 
mapping  and  localization  are  sensor-based  activities.  Mission  planning  concerns  determining 
goals  and  constraints  which  can  then  be  processed  by  the  path  planner;  for  example  that  Robot  1 
should  go  to  Location  A  and  rendezvous  with  Robot  2  by  Time  T.  As  will  be  seen  in  the 
following  sections,  path  planning  relies  on  maps  to  compute  the  path  (usually  as  a  set  of 
waypoints)  based  on  the  mission  goals  and  constraints.  Therefore,  mission  and  path  planning  use 
algorithms  and  approaches  operating  over  knowledge  about  the  situation.  In  contrast  to  mission 
and  path  planning,  mapping  and  localization  are  generative  functions,  where  at  each  step  the 
agent  is  interpreting  sensor  data. 

Path  Planning  Synopsis 

Path  planning  algorithms  for  unmanned  ground  vehicles  has  been  explored  since  the  late 
1960’s.  Approaches  fall  into  two  categories:  topological  and  metric.  Topological  methods  are 
also  known  as  “route”  or  “landmark”  navigation  and  use  topological  maps  or  perceptual 
associations  to  direct  the  robot  to  a  goal.  The  paths  produced  are  usually  tuples  of 
<next_landmark,  movement _strategy>.  Topological  methods  do  not  guarantee  optimality  and 
over  the  years,  their  primary  use  has  been  for  route  retracing.  In  addition,  they  require  a 
projection  of  what  the  robot  would  see  along  the  desired  route;  projecting  perception  from  map 
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data  to  the  degree  of  accuracy  needed  remains  an  open  question.  As  a  result,  this  project  does  not 
consider  topological  methods.  Metric  methods  rely  on  maps  and  usually  produce  a  path  that  is 
optimal  according  to  some  criteria,  usually  distance.  The  paths  produced  are  usually  lists  of 
waypoints,  most  often  in  GPS  coordinates.  The  waypoints  do  not  necessarily  correspond  to  a 
perceivable  landmark  (e.g.,  intersection  of  roads). 
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Figure  4.  TRULLA  Wavefront  Path  Planning  Algorithm 
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Metric  path  planning  algorithms  share  the  same  objectives  and  output  and  also  internal 
representations.  The  algorithms  take  a  volume  of  space  and  distill  it  to  a  more  simple 
representation,  called  the  configuration  space,  such  as  a  Voronoi  graph,  a  regular  grid,  a 
quadtree,  or  hybrid  free  space/vertex  graph.  Given  the  ubiquity  of  maps  which  use  a  grid-based 
coordinate  structure,  regular  grids  are  the  most  common  structure.  The  planning  operation  over 
the  configuration  space  can  be  further  divided  into  graph  or  wavefront  planners. 

Graph  planners  stem  from  the  AI  search  community.  They  convert  the  grid  into  a 
connected  graph  and  use  a  variant  of  the  A*  optimal  graph  search  algorithm  to  find  the  optimal 
path  between  the  start  and  goal  location.  The  edges  between  the  nodes  of  the  graph  represent 
distance.  Obstacles  are  “missing  nodes”  from  the  graph.  Each  graph  node  on  the  path  can  be  a 
waypoint,  though  in  the  earlyl990s  some  nodes  were  eliminated  through  path  relaxation 
techniques  to  smooth  the  path  and  simplify  localization  demands  on  the  perceptual  system.  With 
the  ascendance  of  GPS  for  localization,  path  relaxation  is  now  frequently  skipped. 

Wavefront  planners  originated  from  the  graphics  community.  They  also  use  the  regular 
grid  configuration  space,  but  use  graphics  coloring  algorithms  instead.  These  can  be  thought  of 
as  a  heat  flow  from  the  starting  location  spreading  outward  (coloring)  to  adjacent  cells.  When  the 
“heat”  reaches  the  goal  location,  the  algorithm  is  complete  and  the  shortest  path  can  be  easily 
determined  through  gradients. 

Graph  planners  dominate  unmanned  ground  vehicle  path  planning,  possibly  because  of 
their  use  in  the  DARPA  Autonomous  Land  Vehicle  Project  and  exclusive  use  in  follow  on- 
projects  (UGV  by  the  Army  Research  Laboratory.  The  D*  algorithm  (Stentz,  1995),  which  is  an 
A*  optimal  search  repeated  for  all  possible  starting  points  to  a  goal  is  the  de  facto  standard.  This 
commitment  to  graph  planners  may  have  been  premature  and  have  negatively  impacted  the 
development  of  terrain-sensitive  planners. 

Terrain-sensitive  Path  Planning  Overview 

Terrain-sensitive  path  planning  is  a  newly  emerging  area  within  the  path  planning 
community  that  incorporates  the  impact  of  the  terrain  on  execution  as  well  as  the  shortest 
distance.  Historically,  terrain  has  not  been  a  criteria  for  optimality  for  several  reasons.  First,  until 
the  UGV  Demo  III  project,  there  was  no  demand  to  go  off-road.  Since  the  UGV  was  restricted  to 
roads  or  at  the  worst  case,  open  fields,  sophisticated  terrain  planning  was  not  essential.  Second, 
incorporating  terrain  into  graph  planner  and  maintaining  optimality  is  very  difficult.  D*  and  A* 
planners  require  an  admissible  heuristic  function  to  generate  the  optimal  path  with  the  minimum 
complexity.  Euclidean  distance  is  ideal  because  the  distance  from  a  node  to  the  goal  can  be 
determined  from  the  map.  Admissible  heuristic  functions  with  two  or  more  variables,  in  this  case 
distance  and  traversability,  are  notoriously  difficult  to  construct  without  sacrificing  low  order 
complexity.  Third,  terrain  is  related  to  the  velocity,  turning  radius,  and  traversability  of  a 
particular  vehicle  configuration.  But  often  this  is  a  red  herring  stemming  from  indoor  navigation, 
since  path  planning  outdoors  is  at  the  10M  or  larger  resolution-  far  beyond  the  level  to  worry 
about  turning  radii  and  vehicle  velocity.  Fourth,  terrain  data  is  often  uncertain.  The  level  of 
resolution,  process  of  map  making,  age  of  the  maps,  influence  of  seasons  on  maps,  and  the 
difficulty  in  accurately  projecting  changes  in  foliage  create  problems.  Independently  of  the 
intrinsic  challenges  of  incorporating  terrain  into  path  planning,  most  researchers  have  not  had 
access  to  outdoor  UGVs  and  so  have  not  experienced  the  true  demands  of  the  field. 
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D*  methods  either  ignore  terrain  until  it  becomes  a  problem  during  execution,  prime 
unfavorable  terrain  and  vehicle  configurations  from  the  graph  (e.g.,  it  is  an  obstacle)  during 
preprocessing,  use  time  to  goal  rather  than  distance  to  goal  as  the  metric,  or  a  combination  of  all 
three,  see  (Guo  &  Parker,  2002).  Pruning  eliminates  valuable  options:  going  through  Area  K  may 
not  be  fast  but  at  some  point  may  be  better  than  standing  still.  Using  time  to  goal  appears 
reasonable  though  coarse,  and  also  side-steps  the  hard  problem  of  how  to  convert  the  impact  of 
terrain  on  a  particular  vehicle  to  traversibility  speed. 

Wavefront  planners  appear  better  suited  for  incorporating  terrain  effects.  Instead  of 
treating  terrain  as  an  obstacle,  wavefront  planners  associate  a  density  with  the  terrain.  The  heat 
flow  through  the  area  is  slowed  but  not  arbitrarily  eliminated.  However,  as  with  graph  planners, 
the  impact  of  associating  a  terrain  with  an  accurate  density  or  transit  time  is  still  challenging.  The 
best  known  is  Trulla  (Murphy,  Hughes,  Noll,  &  Marzilli,  1 999),  which  was  developed  by  the 
University  of  South  Florida  and  adopted  by  the  Naval  Research  Laboratory. 

The  key  advantages  of  wavefront  planners  over  graph  planners  are  that  they  incorporate 
terrain  in  the  regular  grid  (essentially,  as  a  map  overlay)  rather  than  in  edge  weights  that  must  be 
separately  maintained  and  computed,  and  that  all  possible  paths  to  all  possible  goals  are  pre¬ 
computed  as  a  side  effect  of  the  wave  propagation  (or,  free  path  planning  that  can  be  cached  on  a 
UGV).  There  are  certainly  alternatives  to  graph  and  wavefront  planners  such  as  the  use  of 
genetic  algorithms  (Farritor  &  Dubowsky,  1997)  and  case-based  reasoning  (Kruusmaa,  2003), 
but  these  are  not  widely  accepted. 

Path  Planning  for  Teams  of  Robots 

A  related  issue  to  path  planning  in  UGVs  is  path  planning  for  teams  of  mobile  robots. 
Only  a  few  approaches  explicitly  consider  the  impact  of  terrain  on  a  team  of  robots  and  in  each 
case,  the  impact  of  terrain  is  handled  during  execution:  the  path  of  the  robots  or  lead  robot  is  first 
computed  and  then  during  execution,  the  robots  adapt.  Approaches  include  considering  the 
robots  as  connected  by  springs  (Lawton,  Beard,  &  Young,  2003),  using  a  control  graph  (Desai, 
2001),  or  Extended  Kalman  Filtering  (EKF)  to  maintain  the  correct  elevation  profile  as  well  as 
formation  (Madhavan,  Fregene,  &  Parker,  2004). 

Terrain  Interpretation  for  Path  Execution  and  Learning 

While  terrain-sensitive  path  planning  uses  a  priori  terrain  knowledge,  the  robot  itself  has 
to  execute  that  path  in  an  open  world  where  expectations  and  plans  may  be  invalidated.  As  a 
result,  the  robot  needs  to  be  able  to  sense  the  current  terrain  and  adapt  accordingly.  Or  in  other 
words,  terrain-sensitive  path  planning  is  a  top-down  process  operating  on  symbolic  data.  A 
bottom-up  learning  process  operating  on  perceptual  data  is  needed  to  update  maps  and  propagate 
successful  navigation  strategies.  We  refer  to  this  as  the  “10m  problem”:  how  to  connect  the 
symbolic  data  that  exists  in  maps  with  10m  x  10m  or  30m  x  30m  resolution  with  the  perceptual 
data  that  the  robot  collects. 

There  are  four  levels  of  adaptation  defining  how  a  robot  would  adapt  to  the  terrain: 

1.  Sensori-motor,  or  behavioral,  changes.  Here  the  robot  would  keep  using  the  active 
behaviors  selected  by  the  TAH-RI  system,  but  would  change  their  parameters.  For 
example,  if  the  terrain  was  “bumpy”  and  causing  problems  with  sensor  stabilization,  the 
robot  would  slow  down. 

2.  Schematic,  or  script-level,  changes.  In  this  case,  the  instantiated  behaviors  are  no  longer 
sufficient  and  so  the  robot  must  change  the  behaviors,  not  tweak  the  parameters.  One 
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example  is  when  the  Cartographer  creates  an  expectation,  such  as  the  estimated  time  to 
arrival,  and  that  expectation  is  not  being  met.  A  subscript  is  instantiated  which  will  then 
request  diagnostics  and  eventually  an  update  of  the  map  and  a  new  set  of  paths,  allowing 
the  robot  to  cope  with  problems.  Another  example  is  changes  in  environmental 
conditions,  where  the  fall  of  night  or  rain  interferes  with  a  video  camera  and  the  system 
adapts  by  switching  to  a  no-light  illuminator. 

3.  Deliberative  changes.  This  level  of  change  begins  to  show  the  power  of  TAH-RI.  As  the 
robot  adapts  its  behaviors  and  parameters  to  the  terrain,  it  learns  the  association  with  this 
terrain  type. 

4.  Distributed  changes.  Here  the  robot  propagates  what  it  has  learned  to  other  robots  or 
works  with  other  robots  to  divide  up  the  computational  task. 

However,  these  four  levels  of  adaptation  presume  the  robot  has  the  ability  to  detect  terrain 
changes.  As  will  be  seen  below,  real-time  terrain  interpretation  is  an  open  question. 

On-board  Terrain  Interpretation 

A  UGV  has  only  two  sources  of  information  about  the  terrain  at  the  10m  or  less 
resolution:  1)  from  a  forward  deployed  source  with  a  limited  look-ahead  such  as  HUMINT,  a 
small  UAV  (Miller,  2002;  Stentz,  Kelly,  Herman,  &  Rander,  2002)  or  another  UGV  or  2)  from 
its  own  onboard  sensor  suite.  However,  forward  or  projective  interpretation  can  be  viewed  as 
onboard  interpretation  that  is  distributed  to  other  agents.  Therefore,  the  body  of  literature  in  real¬ 
time  terrain  interpretation  has  focused  on  onboard  interpretation.  Onboard  interpretation 
approaches  can  be  divided  into  two  categories:  proprioceptive  and  exteroceptive. 

Proprioceptive  Terrain  Interpretation 

Proprioceptive  terrain  interpretation  uses  internal  sensors  such  as  vibration,  wheel  torque, 
and  position  to  determine  terrain.  (Howard,  Seraji,  &  Tunstel  2001;  Larson,  Voyles,  &  Demir 
2004)  use  inclinometers  to  determine  the  slope  of  the  terrain.  (Larson,  Voyles,  &  Demir  2004)  go 
further  and  include  a  measure  of  the  “gait  bounce”  extracted  from  the  video  camera;  essentially 
the  more  the  camera  has  to  compensate  for  visual  servoing  errors,  the  rougher  the  terrain. 
(Iagnemma  and  Dubowsky,  2002)  use  both  vision  and  audition  to  detect  gait  bounce  and  wheel- 
ground  interaction. 

The  advantage  of  proprioception  is  that  it  is  “built  in”  the  robot  and  allows  the  robot  to 
adapt  its  speed  to  the  terrain.  The  disadvantages  are  numerous.  These  methods  are  generally 
limited  to  surface  properties  and  do  not  detect  foliage  changes.  There  is  no  classification  per  se, 
but  rather  these  methods  capture  a  stimulus-response  relationship  between  vehicle  speed  and  the 
salient  terrain  property.  It  is  also  not  particularly  useful  for  detecting  changes  in  symbolic 
expectations  (e.g.,  “why  is  there  an  obstacle  blocking  the  road?”). 

Exteroceptive  Terrain  Interpretation 

Exteroceptive  sensors  perceive  attributes  of  the  environment  external  to  the  robot.  There 
are  three  main  categories  of  exteroceptive  sensors  used  for  terrain  interpretation  on  robots: 
multispectral  imaging,  ladar,  and  color  video.  None  of  these  sensors,  individually  or  combined, 
have  provided  a  reliable  solution  to  terrain  interpretation.  This  suggests  that  for  the  short  term, 
systems  such  as  TAH-RI  which  allow  multiple  levels  of  cues  (“something’s  wrong  here  but  I 
don’t  know  what”)  and  also  the  involvement  of  the  human  in  the  diagnosis  process  will  be 
critical  for  the  successful  deployment  of  UGVs. 
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Multispectral  imagers  have  been  used  on  large  truck  sized  robots  and  have  produced 
good  classification  results,  see  (Bhanu,  Symosek  &  Das,  1 997).  However,  these  sensors  are  very 
large  and  unlikely  to  be  miniaturized  enough  to  fit  on  small  UGVs  within  the  next  5  years. 

Ladar  is  a  much  smaller  sensor  and  has  become  increasingly  popular.  It  is  a  staple  of  the 
Army’s  Demo  III  XUV  program  (Albus  et  al,  2002).  (Castano  and  Matthies,  2002)  used  ladar  to 
detect  dense  foliage  and  trees,  but  had  problems  with  “thin”  trees.  (Hebert,  Vandapel,  Keller,  & 
Donamukkala,  2004)  produced  algorithms  for  generating  3D  models  of  the  environment, 
including  obstacles,  in  real-time.  However,  foliage  remained  a  problem.  (Ollis  &  Jochem,  2003) 
also  used  ladar  for  obstacle  avoidance  and  rudimentary  terrain  analysis.  In  general,  ladar  appears 
useful  for  obstacle  avoidance  and  promising  for  some  types  of  foliage  detection.  It  will  require  a 
great  deal  of  work  to  establish  the  correct  classes  of  perceivable  terrain  as  well  as  deal  with 
obstacles  partially  hidden  by  foliage  (the  rock  behind  the  grass). 

Color  video  is  the  most  popular  sensor  for  extracting  terrain  information,  possibly 
because  color  video  cameras  are  small  and  every  robot  has  one.  There  is  certainly  the  existence 
proof  that  humans  can  discern  terrain  visually.  Many  different  methods  have  been  tried  and  none 
appear  successful  (Davis,  1995;  Lin,  Hays,  Wu,  Kwatra,  &  Liu,  2004;  Dima,  Vandapel,  & 
Hebert,  2003).  Some  improvements  were  made  when  color  video  was  used  in  conjunction  with 
ladar,  see  (Talukder,  2002). 

Findings  Regarding  Terrain-Sensitive  Path  Planning  &  Interpretation 

To  summarize,  graph  planners,  especially  D*,  dominate  path  planning,  but  wavefront 
planners  appear  to  have  significant  advantages  for  terrain-sensitive  path  planning.  The  biggest 
problems  with  current  approaches  stem  not  from  the  algorithms  themselves,  but  rather  from  the 
integration  of  path  planning  into  the  large  navigational  enterprise.  Upstream  of  the  path  planner 
in  the  navigation  process  is  the  map  input.  Path  planners  require  a  priori  maps,  yet  these  maps 
can  be  wrong  and  the  terrain  data  is  uncertain.  In  general  there  is  no  adequate  catalog  of  terrain 
vs.  platform  traversability  characteristics;  all  such  systems  use  ad  hoc  metrics.  Downstream  of 
path  planning  is  execution.  A  path  planner  can  be  optimal  and  correct,  yet  a  UGV  in  the  field 
may  not  be  able  to  execute  the  path  due  to  GPS  area  denial  or  unexpected  terrain.  In  practice, 
there  is  often  no  way  for  the  robot  to  use  its  path  planning  assumptions  with  its  current 
perception  to  detect  expectation  violations.  Finally,  the  “10M  problem”  is  the  most  pervasive. 
Outdoor  path  planning  is  at  a  relatively  coarse,  on  a  10m  grid.  Within  this  10  meter  squared  area, 
numerous  obstacles,  variations  in  terrains,  and  other  navigational  issues  can  arise  that  cannot  be 
captured  in  advance  by  path  planning. 

The  state  of  the  art  in  a  robot  determining  terrain  is  minimal  for  navigation  and  not 
supportive  of  higher-level  reasoning.  This  suggests  that  attention  must  be  paid  to  the  perceptual 
abilities  of  the  robots  and  how  software  and  humans  can  interact  with  the  limited  data  to  produce 
robust  results.  One  observation  is  that  there  is  no  standard  test  bed  or  data  set  for  terrain  by 
which  to  compare  terrain  interpretation  algorithms. 

Task  2  Results:  Determine  What  Tool  Design  Changes  Needed. 

In  Task  2,  our  team  determined  necessary  tool  additions  and  modifications  based  on  the 
results  of  Task  1  for  fielded  command  and  control  systems  like  the  Army’s  FBCB2  and  the 
USMC’s  C2PC,  and  OTB-SAF  simulation  system  used  for  training  and  SMART  investigations 
of  potential  systems  and  new  operational  concepts  in  the  various  BattleLabs.  During  Phase  I 
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several  team  members  were  given  the  opportunity  to  see  FBCB2  in  action,  and  to  get  more 
insight  into  its  underlying  infrastructure  from  that  systems’  developers  at  Northrop-Grumman’s 
Orlando  office.  We  have  reviewed  the  FBCB2  manual  online,  and  reviewed  our  corporate 
experience  with  both  C2PC  and  OTB-SAF. 

Common  Terrain  Features  and  Combinations  of  Other  Terrain  Analysis  Results 

New  functionality  will  need  to  be  added  to  the  Terrain_V5  code  base.  This  new 
functionality  will  include  terrain  feature  (e.g.,  hill  and  ridgeline)  determination  and 
union/intersection  of  geometrically  defined  objects.  All  existing  Terrain_V5/6  module  terrain 
calculations  are  affected  by  different  factors  including  vehicle  type  and  echelon  size.  There  is 
currently  functionality  to  query  for  features  specific  to  a  particular  unit.  We  have  built  an 
architecture  that  allows  us  to  search  for  specific  characteristics  within  an  area  and  overlay  or 
combine  with  other  queries  if  desired,  although  this  capability  should  be  expanded  and  refined. 
For  example,  we  can  query  an  area  for  only  major  roads  that  do  not  go  through  a  city.  The  basic 
operations  use  the  =,  <,  <=  ,  >,  >=  operators  for  searching.  They  produce  values  that  can  be 
combined  using  'and',  'or',  and  'not'.  This  gives  us  a  logical  design  in  querying  terrain  databases 
giving  us  a  very  flexible  design  for  meeting  future  requirements  with  lower  costs  for  expansions 
and  possibly  slightly  lower  costs  for  refinements  of  these  capabilities.  A  key  modification  of 
great  interest  would  be  the  ability  to  modify  and  write  back  to  VPF  as  updated  information  about 
actual  features  and  characteristics  represented  as  f-codes  would  be  quite  useful,  both  for 
individual  robots/humans/units  and  for  sharing  this  info  with  other  friendlies  also  needing  to 
traverse  or  otherwise  reason  about  a  given  area.  This  will  be  a  very  large  task  and  may  be 
beyond  the  scope  of  TAH-RI  Phase  II  without  scoping  it  down  carefully.  It  would  also  be 
helpful  to  be  able  to  use  older  data  formats,  like  DFAD,  which  will  take  some  effort  to  develop. 

Currently,  the  TERRAINV5/6  module  used  in  C2PC  for  the  USMC  has  functionality  to 
cache  vectorized  road  data  for  the  use  of  finding  road  junctions.  In  the  future,  it  should  use  the 
road  vectors  plus  the  junctions  to  create  a  shortest  path  graph  for  all  road  junctions  in  the  area 
taking  into  consideration  the  goodness  of  different  roads  (although  this  again  gets  into  the  area  of 
goodness  for  what  particular  purpose).  Currently  when  the  module  wants  to  get  information 
about  any  area  (e.g.,  Boxy  or  Irregular),  it  has  to  query  and  grab  data  for  a  rectangular  area.  Due 
to  the  fact  that  it  rasterizes  such  data,  over  a  large  area  a  raster  eats  up  a  lot  of  memory  regardless 
of  whether  or  not  the  system  actually  needs  all  that  information.  Thus,  to  handle  irregular 
objects  which  are  essential  to  working  with  real-world  missions  and  commander’s  intent 
information,  we  would  like  to  develop  two  main  algorithms: 

•  Given  two  or  more  polygons,  take  their  union  or  intersection  to  create  a  new  polygon. 

•  Given  a  polygon,  extract  VPF  and  DTED  data  only  for  the  regions  inside  the  polygon. 
To  do  this  we  must  be  able  to  directly  access  the  VPF  data. 

Note  also  that  ensuring  CJMTK  compliance,  where  appropriate,  in  any  additions  or  changes  in 
structure  or  function  of  existing  tools  should  also  be  kept  up. 

Examples  ofTrulla  Terrain-Sensitive  Planning  On-Board  Robot 

In  order  to  illustrate  the  utility  of  the  Trulla  path  planning  algorithm  as  part  of  the  TAH- 
RI  system  and  how  it  fits  within  the  architecture,  Trulla  was  implemented  on  a  RWI ATRV-JR 
mobile  robot  and  three  demonstrations  were  conducted  in  a  field  with  terrain  diversity  and  urban 
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structures.  The  first  two  demonstrations  show  the  nominal  terrain-sensitive  path  planning 
function  and  the  third  shows  the  replanning  in  response  to  a  threat. 

In  each  scenario,  the  Trulla  algorithm  runs  on  the  robot’s  Operator  Control  Unit  (OCU), 
which  is  a  Gateway  laptop.  It  computes  the  set  of  waypoints,  then  transmits  the  list  to  the  robot. 
The  robot  now  has  a  complete  plan  and  reactively  executes  navigation  between  waypoints.  Note 
that  the  Trulla  algorithm  could  reside  directly  onboard  the  robot  or  on  another  distributed  control 
workstation. 
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Figure  5.  Satellite  image  of  the  robot  test  field. 


Example  Test  Area  Terrain 

Figure  5  shows  a  satellite  image  of  the  robot  test  field.  The  darkest  areas  are  trees.  There 
is  an  oak  tree  in  the  lower  left.  A  row  of  four  palm  trees  line  a  sidewalk  extending  from  the  lower 
middle  to  the  lower  right  of  the  image.  A  stand  of  palmetto  bushes  is  above  the  rightmost  palm 
tree.  A  road  is  below  the  sidewalk,  though  not  clearly  visible.  The  field  itself  had  some  terrain 
diversity,  with  grass  and  an  area  with  sand. 

Scenario  1  and  2:  Paths  Around  and  Through  Unfavorable  Terrain 

In  both  scenarios,  the  robot  was  given  starting  and  goal  locations.  Based  on  the  shortest 
distance  and  the  traversibility  and  navigation  constraints,  Trulla  computed  an  optimal  path  which 
the  robot  executed.  The  first  two  scenarios  also  used  the  same  terrain  weighting,  shown  in 
Figure  6,  as  increasing  shades  of  blue.  The  trees  and  shrubs  were  marked  as  obstacles  (highest 
value  of  unfavorable).  The  sand  pit  was  marked  as  mildly  unfavorable.  In  addition,  the  sidewalk 
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was  marked  as  highly  unfavorable  not  because  of  mobility  concerns  but  due  to  constraints 
generated  by  TAH-RI. 

Figure  7  shows  the  output  of  Trulla.  The  start  location  is  in  green  and  the  goal  is  in  red. 
The  white  is  the  optimal  path  with  GPS  waypoints.  The  arrows  show  the  result  of  the  all-paths 
computation-  if  the  robot  deviates  from  the  path,  these  "arrows"  describe  the  alternative  optimal 
path  from  the  new  location.  In  this  case,  Trulla  routes  the  robot  around  the  sand  pile,  since  the 
robot  can  make  better  time  on  grass  than  on  uneven  sand.  Figure  8  shows  robot  on  this  path. 

In  Scenario  2,  the  start  and  goal  locations  were  chosen  so  that  tradeoff  between  avoiding 
less  favorable  terrain  and  moving  to  the  goal  rapidly  was  less.  As  shown  in  Figure  9,  Trulla 
directs  the  robot  to  take  a  shortcut  through  the  sand  pile.  Figure  10  show  the  robot  navigating 
using  GPS  through  the  sand  pit. 
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Figure  6.  Overlay  of  terrain  weighting  on  satellite  image. 
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Figure  7.  Satellite  image  overlaid  with  Trulla  path  and  alternative  paths 


Figure  8.  Robot  navigating  around  sand  pit  in  scenario  1. 
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Figure  10.  Robot  following  path  through  sand  pit. 
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Scenario  3:  Replanning  in  Response  to  Threat 

Scenarios  1  and  2  essentially  duplicate  results  from  previously  published  work,  while 
Scenario  3  shows  new  results  in  using  Trulla  as  a  part  of  the  cognitive  system.  In  this  scenario, 
the  TAH-RI  system  is  informed  of  a  threat,  possibly  an  unmanned  aerial  vehicle  acting  as  a 
sentry  as  in  Figure  11.  The  TAH-RI  system  immediately  reasons  about  the  threat,  changes  the 
map  to  incorporate  new  constraints,  and  generates  a  new  goal. 

In  Figure  12,  it  can  be  seen  that  the  terrain  interpretation  is  now  radically  different.  Open 
areas  are  now  marked  as  unfavorable.  Reasoning  about  the  tree  coverage  and  visibility  of  the 
UGV  from  the  helicopter  produce  a  new  understanding  of  the  environment  (that  the  only  place  to 
hide  is  under  the  oak  tree)  and  forms  a  new  goal.  This  leads  to  the  new  path(s)  in  Figure  13  and 
the  robot  flees  (Figure  14)  to  the  cover  of  the  nearest  tree. 


Figure  11.  Unmanned  aerial  vehicle  conducting  a  search  to  detect  intruders. 


21 


Truth  2.0 


File 


Longitude;  -32  422089  Latitude:  28  0566051 


Coal  Location  — 1 

jcoai  Lore  1-82  42305 
i  Coal  Lac  28  05621 . 


Set  Coal 


current  Location 
i  Current  Lon:  -82-422947] 
|  Current  Lau  28  0563292' 
j .  Tolerance:  0 _ j 

!  |  Set  Current  Location  j 


Workspace 

NW  Lon:  | 

-82.423  I'M  ji 

NW  Lat  i 

28.056606  ji. 

SE  Lorn  i 

-82.422557  J 

SE  Lat  | 

28  0560875$' i 

Feet/Crldeh 

I000434875<$ll 

|  Generate  Workspace  |i 

Options 


Show  Grid  Lines 

□ 

Show  Cell  Weights 

la 

Show  Arrows 

:!0 

Show  Path 

B 

Show  Aerial  Image 

10 

Run  Trulia 


Figure  12.  Different  terrain  interpretation  due  to  threat. 
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Figure  13.  Resultant  goal  and  path  in  response  to  threat. 
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Figure  14.  Robot  being  directed  to  hiding  under  a  tree  by  Trulla  in  response  to  the  threat. 

Discussion  of  Examples  of  Terrain-Sensitive  Planning 

These  demonstrations  show  that  Trulla  is  a  workable  terrain-sensitive  planner.  It  uses  a 
map  representation  that  readily  supports  reasoning  and  propagation  of  constraints.  It  also 
supports  distributed  computing  with  Trulla  being  able  to  reside  on  the  robot  or  a  network  node. 
The  algorithm  does  not  need  to  continuously  recompute  the  path  based  on  the  robot’s  actual 
location  in  order  to  be  optimal  during  execution  since  it  computes  the  best  path  from  all  possible 
locations  as  a  side  effect.  It  can  recompute  paths  in  real-time  given  new  constraints  or  situations. 
Coupled  with  GPS  localization,  robots  can  avoid  unfavorable  terrain  that  they  cannot  themselves 
interpret  in  real-time.  Figure  8  shows  the  robot  navigating  around  the  sand  pit  without  having  to 
sense  it.  Given  the  problems  with  onboard  terrain  interpretation,  this  feature  is  an  advantage. 

The  demonstrations  did  not  show  some  additional  key  features  of  Trulla  that  will  be 
demonstrated  in  Phase  II.  First,  Trulla  provides  the  robot  with  all  possible  paths,  not  just  the 
waypoints.  If  an  unmodeled  obstacle  introduces  a  path  deviation,  the  deviation  may  actually  be 
favorable  and  lead  to  a  short  cut.  Rather  than  have  the  robot  go  to  the  original  waypoint,  Trulla 
will  direct  the  robot  to  the  most  optimal  path  without  replanning.  Second,  Trulla  uses  the  dot 
product  of  the  current  path  with  the  planned  path  to  detect  when  the  robot  had  wandered  off 
course,  most  likely  due  to  a  series  of  unmodelled  obstacles  or  an  error  in  the  map,  and  request 
map  updating  and  replanning.  This  feature  was  not  implemented  in  Phase  I. 

Task  3  Results:  Investigate  Use  of  Commander’s  Mission  Intent. 

CHI  Systems  has  significant  experience  with  this  problem  and  with  a  variety  of  partial 
solutions  that  have  been  attempted  in  the  past.  Truly  solving  this  problem  requires  a  mixture  of 
old  and  new  approaches,  and  it  is  quite  possible  that  the  state  of  the  art  in  key  technical  areas 
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(e.g.,  intelligent  software)  may  not  yet  be  far  enough  advanced  to  make  a  full  solution  possible. 
What  makes  commander’s  intent  such  an  interesting  problem  is  that  current  oral  or  written 
statements  of  commander’s  intent  seem  to  often  focus  primarily  on  providing  additional  insight 
into  the  commander’s  values  as  applied  to  commander’s  assessment  of  range  of  anticipated 
potential  situations  and  how  they  are  expected  to  unfold,  other  possible  ways  in  which  they 
might  alternatively  unfold,  and  guidance  as  to  the  commander’s  intentions  about  how  his  forces 
are  to  position  themselves  to  execute  that  commander’s  plan  as  envisioned  while  ensuring  that 
anticipated  alternative  ways  the  battle  situation  might  unfold  are  covered  using  commander’s 
values,  situated  beliefs,  judgments,  and  tactical  decisions.  In  large  part,  a  key  value  of 
commander’s  intent  seems  to  be  the  communication  to  subordinates  of  an  elaboration  of  how  the 
mission  and  associated  plan  and  alternative  plans  and  pre-planned  responses  are  based  on  the 
commander’s  beliefs,  values,  judgment,  and  expectations  as  each  are  applied  to  the  range  of 
anticipated  possible  situations  that  might  occur. 

Interestingly,  advances  over  the  past  few  years  in  intelligent  agent’s  abilities  to  provide 
humans  with  explanations  of  agent’s  actions,  plans,  and  the  reasoning  underlying  them  may  be 
closely  approximating  commander’s  intent  statements.  How  close  a  match  they  are  is, 
unfortunately,  still  an  open  question.  In  some  of  the  authors’  opinions,  the  match  is  quite  close. 
To  many  AI  researchers  and  cognitive  scientists  and  engineers  studying  that  problem,  what  their 
agents’  have  to  do  to  generate  explanations  seems  to  be  well-covered  in  the  above  description  of 
an  intent  statement  covering  a  commander’s  situated  beliefs  about  the  current  and  anticipated 
situations,  and  how  the  commander’s  values  are  to  be  applied  to  battlefield  decision  making 
under  the  current  and  anticipated  situations  by  subordinates,  as  well  as  how  these  commander’s 
intent  statements  further  elaborate  how  planned  strategies,  operations,  and  tactics  are  to  be 
applied  and  executed  as  the  situation  unfolds. 

Commander’s  intent  must  be  captured  from  human  commanders  in  a  way  in  which  both 
robots  and  human  subordinates  understand  it,  and  can  reason  about  alternative  solutions  when 
faced  with,  in  the  example  of  path  planning,  an  unforeseen  obstacle  or  threat.  General  HRI 
issues  underlying  successful  communication  of  the  commander’s  intent  are  addressed  in  the 
discussion  section,  but  details  of  GUI  design  criteria  to  be  applied  in  prototyping  and  then 
building  a  GUI  to  accomplish  this  challenging  task  are  described  below. 

Current  digital  C2  tools  (and  especially  simulation  systems  like  OTB)  provide  little  or  no 
support  (other  than  some  form  of  instant  messaging  “solution”)  for  enabling  a  commander  to 
elucidate  his  beliefs  about  which  of  the  many  possible  enemy  alternative  courses  of  action 
(COAs)  he  judges  to  be  worthy  of  including  in  contingency  planning,  and  the  details  of  how  he 
wants  to  handle  such  contingencies  if  they  arise.  Also  missing  are  means  of  capturing  key 
guidance  about  value  judgments  and  how  they  should  apply  to  envisioned  contingencies  and 
unanticipated  situations  or  evolutions  of  situations.  However,  when  it  comes  to  capturing  how 
the  enemy  will  most  likely  behave,  and  planning  own  forces  operations,  current  digital  tools  do 
provide  useful  means  of  at  least  creating  and  sharing  representations  of  the  various  COAs  (real 
or  imagined)  through  graphical  overlays  of  maneuver  graphics  and  similar  representations. 
Additionally,  means  of  capturing,  storing,  and  sharing  details  of  the  synchronization  matrix  for 
current  planned  (and  alternative)  COAs  is  possible  on  at  least  some  of  these  tools  or  near-future 
prototypes.  Thus,  it  may  be  possible  to  provide  full  GUI  support  for  capturing  commander’s 
intent  by  augmenting  such  representations  with  elucidation  of  value  judgments,  relative  merits 
and  special  dangers  posed  by  each  alternative,  the  commander’s  preferred  tactics,  techniques, 
and  procedures  (TTP),  and  how  they  all  apply  to  various  envisioned  situations  and  plans.  Due  to 
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the  vital  importance  of  commander’s  intent  for  enabling  autonomy,  more  work  is  needed  to 
prototype  and  test  means  of  advancing  the  state  of  the  art  in  this  area. 

Task  4  Results:  Assess  How  to  Integrate  CROP  with  Robots. 

Our  team  has  completed  our  investigation  of  this  issue  and  the  related  issue  of  which 
details  and  which  analyses  need  to  be  done  on-board  an  autonomous  robotic  system  vs.  off-board 
through  use  of  disembodied  intelligent  software  agents  as  surrogates  for  their  embodied  cousins 
but  with  access  to  significantly  more  computational  power  to  aid  in  integrating  these  analyses 
results  and  data  sources.  The  results  of  our  assessment  have  been  folded  into  refinements  of  our 
initial  design  concept  for  the  underlying  TAH-RI  sub-assembly  architecture  addressed  in  Task  7, 
below. 

A  Multi-Resolution  Relative  Location  Grid  (Multi-Grid)  Spatial  Representation 

To  effectively  reason  about  METT-T  factors,  battlespace-aware  robots  need  means  of 
integrating  CROP  with  terrain  understanding  to  enable  both  better  navigation  and  mission 
performance.  Effective  implementation  of  such  a  capability  will  require  that  a  number  of 
technical  obstacles  be  overcome.  Issues  which  have  currently  been  identified  exist  in  the  areas  of 
methods  of  data  aggregation,  storage,  entity  notification  and  efficient  methods  of  recalculation. 

Path  planning  and  decision  making  using  TAH-RI  will  require  that  data  from  a  number  of 
sources  be  aggregated  and  stored.  Data  sources  that  are  expected  to  be  used  include  terrain 
analysis  results,  direct  fire  information  for  known  weapon  systems,  indirect  fire  information  for 
known  weapon  systems  and  known  unit  position  and  associated  unit  parametric  data.  These  data 
can  be  further  broken  down  into  four  broad  areas  of  focus:  personal,  group  personal,  non- 
associated  individual  and  non-associated  group.  Appropriate  aggregation  of  disparate  data  types 
will  be  based  on  results  of  analyses  of  established  military  practice  and  Soldier  expertise. 

For  purposes  of  data  retrieval  and  analysis,  aggregated  data  will  be  used  within  TAH-RI 
decision-making  components  in  a  multi-resolution  grid  format.  This  data  format  involves 
increased  data  granularity  as  distance  from  focus  (e.g.,  “my  current  location”)  is  increased,  as 
shown  in  Figure  15.  Figure  16  illustrates  how  size  of  minimum  grid  element  (i.e.,  scale  of  Multi- 
Grid)  affects  coverage  and  resolution  of  Multi-Grid  spatial  representation  and  query  tool. 
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Figure  15.  Multi-Resolution  Relative  Location  Grid  (Multi-Grid)  Spatial  Representation 
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Figure  16.  Example  Multi-Grid  Coverage  Areas  at  Five  Different  Scales 


An  efficient  local  addressing  scheme  for  data  stored  in  this  format  must  be  developed. 
Furthermore  this  addressing  scheme  must  allow  for  efficient  conversion  to  a  global  addressing 
scheme  for  effective  data  caching.  An  outline  of  a  reasonable  local  addressing  scheme  follows: 

[ring  :  ring  element :  sub-element] 

Where  ring  is  the  number  of  rings  desired  element  falls  from  the  center,  ring  element 
addresses  one  of  the  eight  grid  squares  in  the  ring  (e.g.  E  is  0,  SE  is  1,  S  is  3,  etc.),  and  sub¬ 
element  address  refers  to  one  of  the  nine  sub-elements  of  the  selected  grid  square.  Using  this 
addressing  method  will  allow  for  easy  description  of  areas  of  tactical  interest.  It  will  also  be 
necessary  to  develop  routines  to  convert  a  C3Core  World_Point  to  an  address  in  the  local  system 
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and  to  convert  local  addresses  to  a  global  addressing  scheme.  Conversion  of  World_Point  to 
local  addressing  should  be  fairly  easy  to  accomplish.  Conversion  in  the  other  direction,  from 
Local  to  Global  addressing  will  require  a  little  more  work  as  the  number  of  different  possible 
offsets  increase  exponentially  with  the  ring  aggregation  level. 

Importantly,  note  that  the  rectilinear  grid  system  depicted  above  could,  in  a  more 
advanced  form,  could  be  composed  of  more  meaningful  “concentric  areas”  that  have  specific 
tactical  meanings.  For  instance,  ring  0  could  be  defined  as  having  its  center  at  a  robot’s  current 
location  and  an  outer  boundary  based  on  how  far  that  particular  type  of  robot  might  be  expected 
to  move  in  some  specific  period  of  time  (e.g.,  5  minutes).  Note  that  as  terrain  impact  on  mobility 
is  factored  into  this  calculation,  this  outer  boundary  becomes  deformed  into  an  irregular  polygon 
surrounding  the  robot’s  current  position.  Moving  uphill  on  a  scree  (i.e.,  loose  rocks)  slope  will 
force  the  intrepid  robot  to  travel  much  more  slowly  (if  at  all)  than  on  relatively  flat  solid  ground 
with  no  impediments.  More  interestingly,  one  could  define  another  ring  further  out  from  an 
armed  robot’s  current  location  with  an  outer  boundary  as  the  robot’s  own  internal  sensors’ 
maximum  range  under  current  conditions  (constrained  by  observability  impact  of  terrain),  and  an 
inner  boundary  as  the  robot’s  on-board  direct  fire  weapons’  maximum  range  in  each  direction 
under  current  conditions  (constrained  by  the  terrain  limitations  on  fields  of  fire.).  Each  time  the 
robot  would  access  and  assess  the  tactical  situation  for  navigation  and  mission  performance 
purposes,  the  particular  shape  and  size  of  this  “ring”  would  be  freshly  calculated  and  used  to  help 
interpret  and  make  decisions  about  the  tactical  situation  based  on  current  CROP  data.  TAH-RI 
could,  theoretically,  be  used  in  such  a  mode  aboard  a  larger  advanced  robotic  vehicle,  although 
the  computational  constraints  involved  in  recalculating  this  kind  of  dynamic  spatial 
representation  would  likely  be  prohibitive. 

In  addition  to  an  appropriate  addressing  scheme,  it  will  also  be  necessary  to  provide  for  a 
two-tiered  event  notification  system.  Specifically,  it  will  be  necessary  to  have  entities  notified 
when  other  entities  of  interest  come  within  a  set  distance  of  them.  This  notification  will  then  alert 
the  entity  that  it  will  want  more  detailed  notification  of  the  unit  of  interest’s  movements  until  the 
point  at  which  that  unit  moves  out  of  range  again.  More  detailed  notification  will  consist  of 
notification  whenever  the  unit  of  interest  crosses  a  cell  boundary. 

In  order  to  accommodate  data  presented  in  this  format,  frequent  data  aggregations  will  be 
necessary.  A  shift  of  one  minimum  size  cell  realigns  the  boundaries  of  all  larger  sizes  of  cells.  In 
order  to  mitigate  the  time  cost  of  recalculating  on  every  boundary  crossing  a  number  of  strategies 

may  be  adopted.  Caching  of  previously  calculated  data  would  ease  computation  in  the  event  of 
backtracking.  Incremental  subtractions  and  additions  to  larger  cells  would  ease  some 
computation  as  well.  If  this  method  is  chosen,  care  will  have  to  be  taken  so  that  errors  don’t 
creep  in  over  time.  Errors  of  this  sort  may  be  avoided  by  requiring  a  complete  refresh  after  a 
certain  number  of  moves.  If  large  periods  of  inactivity  are  expected,  cell  values  of  moves  within 
a  given  limit  may  be  pre-calculated. 

Task  5  Results  :  Determine  What  Mixed-Initiative  Communications  are  Required. 

A  literature  search  has  been  completed,  cataloging  relevant  mixed-initiative 
communications  issues  and  approaches  applicable  to  the  issues  addressed  in  this  task. 
Requirements  analysis  based  on  the  results  of  that  literature  search  were  completed,  and  their 
impact  on  GUI  design  and  on  overall  TAH-RI  architecture  has  been  incorporated  into  the  TAH- 
RI  architecture  results  (Task  7)  and  the  Human-Robot  Interaction  (HRI)  discussions. 
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Mixed  initiative  architectures  are  needed  to  support  flexible  human-robot  teams  capable 
of  responding  to  the  changing  interaction  requirements  related  to  accomplishing  tasks  in 
challenging  environmental  conditions  (Adams,  Rani,  &  Sarkar,  2004;  Bruemer,  Marble, 
Dudenhoeffer,  Anderson,  &  McKay,  2002).  Mixed  initiative  systems  can  support  a  variety  of 
control  levels,  the  implication  being  that  both  humans  and  robots  will  need  an  understanding  of 
the  conditions  under  which  humans  and  robots  gain  and  relinquish  control,  and  how  that 
transition  is  executed  during  terrain  analysis  tasks,  this  will  be  described  further  under  ‘Human- 
Robot  Interaction  with  TAH-RI’  in  the  Discussion  section. 


Task  6  Results  :  Assess  Requirements  for  Real-Time  User-Updatable  Database(s). 

In  this  task,  CHI  Systems  drew  upon  our  experience  with  the  C2PC  system  and  with  our 
own  C3DB  software  component,  which  is  expected  to  form  the  basis  for  the  TAH-RI  real-time 
user-updatable  database  of  terrain  information  and  analysis  results,  as  well  as  for  being  the 
means  of  communicating  the  CROP  to  individual  humans  and  robots.  Given  the  requirements 
and  implications  of  design  decisions  made  in  other  tasks,  it  appears  that  this  anticipated  approach 
will,  indeed,  provide  the  most  capable  approach  available  for  solving  this  part  of  the  terrain 
understanding  problem.  This  assessment  also  affected  our  Multi-Grid  design  reported  above  in 
Task  6  results. 


Task  7:  Complete  the  TAH-RI  Architecture. 

TAH-RI  architecture  design  is  at  the  core  of  Phase  I  design  and  feasibility  assessment. 

As  a  first  step,  based  in  part  on  the  literature  review  from  Task  1 ,  our  USF  colleagues  have 
completed  encapsulation  of  the  TRULLA  terrain-based  wavefront  planner  as  a  separate  module 
and  have  integrated  it  onboard  the  ATRVjr  robot  designated  for  this  Phase  I  effort,  as  described 
above.  Initial  Phase  I  design  for  integration  of  TAH-RI  components  using  CHI  Systems’  C3 
Architecture  Framework  (C3AF)  was  also  completed  in  this  task.  Determination  of  how  C3AF 
and  the  C3Core  component  technologies  based  on  it,  the  iGEN®  cognition  engine,  and  the 
Distributed  Field  Robot  Architecture  (DFRA)  should  be  further  integrated  in  Phase  II  was  a  key 
activity  in  Phase  I,  including  integration  of  the  important  results  emerging  from  our  joint  work 
on  Task  4.  Descriptions  of  key  enabling  technologies  to  be  used  in  Phase  II  implementation  of 
TAH-RI  and  their  relationships  with  the  other  results  discussed  in  this  report  are  provided  below, 

C 3  Core  Component  Technology 

A  key  enabler  for  creating  a  viable  TAH-RI  sub-assembly  is  C3Core,  CHI  Systems’ 
COTS  command  and  control  software  component  technology.  C3Core  is  an  object  oriented, 
service-based  software  suite  that  allows  developers  to  pick  and  choose  a  variety  of  command, 
control  and  battlefield  visualization  functions  for  reuse.  Design  tenets  underlying  C3Core 
software  include: 

•  Core  Sets  of  Reusable  Services.  Components  of  C3Core  can  be  updated,  redefined, 
and  easily  specialized  to  create  software  which  can  operate  within  a  variety  of 
platforms  and  domains. 

•  Platform,  Operating  System,  and  Technology  Independence.  A  goal  in  the 
development  of  C3Core  is  to  maintain  platform,  operating  system  and  technology 
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independence.  Currently  ,the  C3core  operates  natively  under  Microsoft's 
Win95/98/NT,  Solaris  2.4  and  2.5x,  Silicon  Graphics  Irix  6.x  and  Linux.  An 
application  written  utilizing  some  of  the  C4I  component  ware  provides  a  default 
interface  if  required,  and  can  also  run  without  a  GUI  as  an  embedded  decision  aid 
with  a  third  party  interface  such  as  a  Crusader  howitzer  or  Ml  Tank. 

•  No  Assumption  on  Who  Uses  Services.  The  third  property  of  the  C4I  component 
ware  is  that  very  few  assumptions  are  made  regarding  to  whom  services  will  be 
provided,  or  how  services  will  be  provided.  This  requires  that  the  internal  system 
depend  upon  only  abstract  service  interfaces  and  not  on  a  specific  implementation. 
This  does  not  say  that  all  Service  Groups  or  components  are  completely  independent 
of  one  another;  there  exists  a  conceptual  abstract  dependency  within  the  services.  For 
example,  a  graphics  package  needs  to  have  a  context  from  which  to  draw;  this  is  a 
dependency  which  "makes  sense."  The  graphics  package  does  not  care  however  if 
the  context  is  a  printed  page,  3D  display  or  an  analysis  package. 

Figure  17  gives  an  overview  of  where  the  C3Core  component  technology  has  been  applied  in 
BattleLab  experiments  and  demonstration  of  future  technologies  at  Echelons  Brigade  and  Below. 


Figure  17.  Variety  of  Scalable  Composable  Application  Environments  for  C3Core 

Components 


C3Core  was  used  in  the  development  of  the  Combat  Decision  Aiding  System  (CDAS) 
technology  demonstration  testbed.  CDAS  Objective  Force  Warrior  (OFW)  is  described  below  as 
it  provides  one  example  of  an  existing  digital  C2  system  that  could  be  enhanced  using  TAH-RI, 
while  also  illustrating  development  using  CHI  Systems’  existing  C3  Architecture  Framework 
(C3AF)  core  and  the  C3Core  reusable  software  components  built  atop  that  core. 

Example  Application  of  CSCore  Component  Technology  for  C2  System 

CDAS  OFW  provides  multi-echelon  netted  fires  capability,  from  one  individual  OFW 
equipped  dismounted  Soldier  up  to  a  whole  Unit  of  Action  Effects  Control  Center  (see  Figure  1 8 
for  the  “combat  view”  CDAS  interface).  CDAS  OFW  facilitates: 

•  Collaborative  Planning, 

•  Logistics  Monitoring  and  Asset  Visibility, 
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•  Terrain  Analysis  for  Sensor  Placement, 

•  Issuing  Digital  Orders, 

•  Use  of  Robotic  Vehicles, 

•  Text  Messaging  (Instant  Messenger),  and 

•  Situational  Awareness  Monitoring. 

An  experiment/demonstration  was  conducted  at  Fort  Benning,  GA,  examining  whether  the 
individual  OFW  Soldier,  fire  team  leader,  and  squad  leader  could  reach  back  quickly  to  obtain 
support  from  Non-Line-of-Sight  (NLOS)  and  Beyond-Line-of-Sight  (BLOS)  weapon  systems, 
whether  leaders  could  perform  distributed  interactive  fires  management  of  squad-level  weapon 
systems,  and  the  effects  that  could  be  expected  from  improving  situational  awareness  for  the 
dismounted  leader  and  Soldier. 


Figure  18.  CDAS  Combat  View  Interface 

Several  findings  were  reported  from  the  experiment/demonstration.  Overall,  the  results 
were  successful,  illustrating  that  a  decision  support  tool  can  improve  performance  of  a 
dismounted  Soldier  in  the  field.  A  few  examples  of  the  results  are: 

•  Each  of  the  Soldiers  who  participated  in  the  experiment  was  able  to  call  for  fire  using 
the  OFW  CDAS.  By  a  large  majority,  the  participants  agreed  that  sending  the  call  for 
fire  using  OFW  CDAS  was  easier  than  sending  a  voice  call  for  fire. 

•  The  average  for  single  mission  times  sent  from  a  Soldier  was  23  seconds  from  the 
sending  of  the  call  for  fire  to  the  time  when  the  weapon  system  fired  within  the 
simulation  environment,  a  dramatic  improvement  over  current  mission  processing 
times  for  targets  of  opportunity:  2.5  minutes  in  Paladin  Mission  Training  Plan. 

•  The  OFW  CDAS  self-oriented  map  display  was  used  extensively  by  all  of  the 
participants.  This  allowed  the  Soldier  to  navigate  the  urban  complex  where  the 
experiment  took  place,  with  the  map  moving  and  rotating  to  account  for  the  Soldier’s 
direction  of  travel  and  location,  and  also  showing  friendly  and  enemy  locations.  All 
participants  desired  to  have  this  capability  in  real  world  applications. 
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•  Ease  of  use  and  rapid  training  are  essential  elements  for  success  of  any  OFW  system 
used  by  the  dismounted  Soldier.  Based  on  the  preliminary  results  of  this 
experiment/demonstration,  the  OFW  CDAS  appears  to  be  moving  in  the  right 
direction  concerning  usability  and  training.  Training  could  be  accomplished  in  less 
than  one  hour,  and  for  some  individuals  in  less  than  1 0  minutes. 

Building  /  Enhancing  Systems  with  CSCore  Component  Technology 


There  are  many  advantages  to  reusing  existing  C3Core  component  parts  for  developing 
systems.  Figures  19  and  20  illustrate  some  of  those  advantages  for  TAH-RI  development.  CHI 
Systems  currently  (2004  Catalog  in  effect  at  start  of  Phase  I)  offers  33  separate  C3Core 
component  parts,  against  which  the  functional  requirements  for  TAH-RI  were  matched  to 
identify  appropriate  component  parts  that  could  be  used  to  support  this  effort. 


Building  Applications  with  C3Core’s  Component  Based  Architecture 
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Figure  19.  Why  Buiid  Applications  with  C3Core  Components? 
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Figure  20.  TAH-RI  Architecture  To  Be  Built  With  C3AF  Components 


The  whole  catalogue  of  C3Core  components  available  when  this  Phase  I  effort  began  are 
described  below  in  Table  I,  and  pertain  to  a  wide  range  of  C4ISR-related  capabilities,  including: 
imagery,  static  data,  dynamic  data,  terrain  analysis,  decision  aiding,  mobility  analysis,  input, 
planning,  and  situation  awareness.  It  is  expected  that  many  of  these  components  could  be 
applied  “as  is”  for  Phase  II  development  of  TAH-RI  and  for  enhancement  of  existing  C2  digital 
tools  with  the  TAH-RI  sub-assembly.  Other  components  may  be  modified  and  some 
requirements  may  not  be  represented  in  the  current  software  suite.  Technical  requirements  for 
these  new  components  addressing  unmet  needs,  will  have  to  be  assessed  and  the  new 
components  can  then  be  designed  and  developed  as  soon  as  practicable  (for  TAH-RI 
development,  this  will  be  the  principal  focus  of  Phase  II  development).  This  will  narrow  the 
focus  of  the  development  effort  to  be  conducted  in  Phase  II  largely  to  those  unfilled 
requirements,  which  must  be  added  to  the  C3Core  component  parts  software  collection.  Each 
component  comes  with  a  detailed  technical  description,  including  Application  Programmer 
Interface  (API)  documentation,  and  can  be  adapted  as  needed  for  the  proposed  systems’ 
functional  description. 


Table  I  Reusable  C3Core  Software  Components  Based  on  C3AF  (2004  Catalog) 

Function 

Description 

Imagery 

ADRG  Translator.  Extracts  the  raw  data  which  is  in  an  ADRG  file  format 
and  produces  data  files  in  a  tiled  format  suitable  for  mass  storage. 

Map  Image  Builder.  Provides  the  management  capabilities  reauired  to  build 
generic  raster  format  map  images. 

Tactical  Symbol  Generator.  Provides  the  capability  to  produce  MIL-STD- 
2525  unit  symbology  based  upon  Force  Code,  Echelon,  Platform,  and 
Organization. 

Alert  Log.  Provides  a  general  service  to  allow  for  the  collection,  retrieval, 
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Table  I  Reusable  C3Core  Software  Components  Based  on  C3AF  (2004  Catalog) 

Function 

Description 

(TAH-RI: 

HRI) 

persistent  storage  and  dissemination  of  alerts  or  user  notifications. 

E-Mail  Component.  Provides  e-mail  based  inter-operability  between  remote 
applications  and  is  applicable  through  all  domains  and  echelons 

Static  Data 

(TAH-RI) 

Unit  Knowledge  Base.  Provides  fundamental  properties  of  combat  units,  such 
as  weapons  systems,  weapons  ranges,  and  mobility  speeds,  for  example. 

DTED  Manager.  The  DTED  manager  component  provides  a  simple  interface 
to  digital  elevation  data  in  NIMA's  DTED  format. 

VPF  Extractor.  Allows  the  extraction  of  data  that  is  in  the  VPF  (MIL-xxx) 
format. 

Dynamic 

Data 

(TAH-RI) 

User  Terrain  Categorization.  Allows  clients  to  describe  terrain  categorization 
in  a  collaborative  fashion  which  can  be  used  to  overlay,  augment  or  override 
terrain  categorizations  based  upon  static  data.  This  is  typically  utilized  to  store 
transient  terrain  anomalies  such  as  flooded  terrain  or  contaminated  areas. 

Object  Data  Store.  Provides  an  easv  to  use  API  to  allow  for  the  definition  and 
description  of  objects  which  are  either  planned  or  actual  and  can  form  the  basis 
of  an  application’s  situational  awareness. 

Terrain 

Analysis 

(TAH-RI) 

DTED  Categorization.  Produces  a  terrain  categorization  for  unit  mobility 
based  either  upon  user  defined  parameters  or  default  values  for  either  a 
directional  or  non-directional  analysis. 

Cover  &  Concealment  Evaluation  (C&C).  Provides  either  a  cover  or 
concealment  evaluation  upon  a  user  defined  area  or  point  location  using 
standard  VITD  data. 

Generic  Categorization.  Provides  clients  a  high  level  abstraction  above 
individual  terrain  analysis  components  and  provides  automated  data  fusion  of 
these  sub-components.  This  component  is  applicable  within  any  environment 
when  different  types  of  terrain  analysis  must  be  performed  and  the  results  fused 
into  a  homogenous  product  for  post  processing  or  display. 

Elevation  Fusion.  Combines  static  DTED  and  VITD  data  together  with  user 
elevation  data  to  produce  composite  elevation  maps. 

Line  of  Sight  (LOS).  Performs  both  point  and  area  LOS  calculations  based 
either  upon  raw  elevation  data  or  composite  elevation  data  based  upon  the 
Elevation  Fusion  component. 

VITD  Categorization.  Performs  terrain  categorization  based  upon  VITD 
(Vector  Interim  Terrain  Data)  based  upon  three  primary  types  of  unit 
composition:  Light  Infantry,  Armor/Mech  and  M1/M2.  All  terrain  data  for  a 
given  user  defined  area  is  passed  through  an  extensive  set  of  rules  which 
categorizes  the  terrain  into  5  different  categories;  Unrestricted,  Restricted, 
Severely  Restricted,  Super  Severely  Restricted  and  Impossible. 
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_ Table  I  Reusable  C3Core  Software  Components  Based  on  C3AF  (2004  Catalog) 

Function  Description 

Other  Alerting  Decision  Aid.  Assists  in  the  monitoring  and  management  of  cautions, 

Decision  warnings,  and  advisory  messages  from  all  decision  aiding  components  in  the 

Aiding  weapon  system.  Currently,  the  ADA  is  intended  to  manage  alerts  from  the 

following  decision  aiding  systems:  NBC;  Maintenance;  Logistics/sustainability; 
Self  Protection;  Route  Planning;  Battle  Position  Selection;  and  Planning.  Other 
decision  aids  can  be  easily  added. 

Maintenance  Decision  Aid.  Assists  in  the  monitoring  of  combat  vehicle 
maintenance  status,  planning  of  scheduled  and  unscheduled  maintenance,  and 
performing  emergency  battle  damage  assessment  and  repair. 

Logistics/Supportabilitv  Decision  Aid.  Assists  the  user  of  the  Artillery 
Decision  Aid  in  planning  of  vehicle  logistics  resupply  and  shortfall  prediction; 
and  monitoring  of  the  current  vehicle  logistics  situation  during  combat 
operations. 

NBC  Defense  Decision  Aid  (NBCDAY  Assists  the  user  in  monitoring 
predicted  and  actual  NBC  conditions  during  combat  operations,  and  to  aid  the 
user  to  develop  passive  and  active  defense  and  avoidance  measures  against 
NBC  threats. 

Mobility  Mobility  Corridors.  Identifies  high  speed  mobility  corridors  though  a  piece  of 

Analysis  terrain. 

(TAH-RI)  Alternate  Routes.  Fills  the  gap  between  optimal  route  selection  and  plausible 
route  selection.  A  client  specifies  an  area  for  analysis  and  many  different  useful 
calculations  are  performed  to  help  identify  or  predict  mobility  behavior  through 
terrain. 

Mobility  Range  Analysis.  Produces  different  types  of  movement  analysis 
through  terrain. 

Covered  Routes.  Selects  mobility  routes  through  terrain  which  optimize  both 
speed  and  coverage. 

Position  Selection.  Assists  in  the  selection  and  evaluation  of  a  firing  location 
for  either  direct  or  indirect  weapons  platforms. 

Route  Distribution  Model.  Determines  high  plausible  area  in  the  terrain 
which  an  object  might  travel.  This  form  of  calculation  is  extensively  utilized  for 
identification  of  Named  Areas  of  Interest  (NAI)  and  Targeted  area  of  Interest 
(TAI),  which  can  help  in  the  development  of  Sensor  Management  plans  and 
Fire  Mission  Plans. 

Battle  Position  Selection.  Assists  in  the  selection  and  evaluation  of  a  firing 
location  for  either  direct  or  indirect  weapons  platforms. 

Input  Eve  Tracker.  Allows  users  to  interact  with  the  application  using  visual 

gestures  as  opposed  to  typical  mouse  control  for  hands  free  operation.  When 
combined  by  a  voice  recognition  component,  this  component  allows  true, 
hands-free  operation  of  the  computer’s  graphical  user  interface. 
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Table  I  Reusable  C3Core  Software  Components  Based  on  C3AF  (2004  Catalog) 

Function 

Description 

Planning 

(TAH-RI: 

HRI) 

Mission  Planner.  Allows  clients  to  create,  manipulate,  disseminate  and  store  a 
planning  scenario  or  course  of  action.  This  can  be  effectively  utilized  for  the 
generation  of  mission  orders,  war  gaming  or  simulation  during  the  planning 
process.  During  the  execution  phase  of  a  mission  the  planning  component  can 
seamlessly  integrate  with  situational  awareness  object  store  to  provide  the  basis 
for  alerting  mechanisms  or  decision  aids. 

Static  Plan  Checker.  Associated  with  the  Planning  component  to  provide  an 
automated  means  to  validate  plans  prior  to  the  development  of  mission  orders. 
Dynamic  Plan  Checker.  Provides  plan  validation  in  regards  to  feasibility  and 
optimality,  analyzing  whether  a  plan  is  technically  feasible  within  the  client 
provided  constraints. 

Planning/ 

Situational 

Awareness 

(TAH-RI: 

HRI) 

Run  Time  Alerting.  Similar  to  both  the  Static  and  Dynamic  Rule  components 
though  the  rules  are  tailored  towards  detecting  differences  between  the  plan  and 
the  current  situation.  Three  different  rules  are  incorporated  into  this  component, 
which  categorize  the  type  of  alert  based  upon  threat. 

Synchronization  Matrix.  Produces  a  time,  space  and  action  overview  of  a 
plan  or  operation  which  closely  integrates  within  the  IPB  or  situational 
awareness  actions  currently  utilized  in  the  military. 

Process 

Control 

(TAH-RI) 

Event  State  Machine.  Provides  an  interface  for  3rd  party  applications  to 
become  part  of  the  control  event  loop  such  as  a  GUI  environment. 

In  Phase  II  development,  CHI  Systems  technical  team  will  identify  the  intended  hardware 
footprint(s)  for  the  proposed  system  and  ensure  that  the  software  to  be  adapted  for  TAH-RI  is 
scaled  to  the  anticipated  footprint(s).  Code  profiling  and  rigid  adherence  to  best  practices  for 
coding  during  implementation  are  the  principal  means  of  ensuring  a  good  fit  to  the  intended 
delivery  platform.  C3AF  and  most  of  the  C3Core  component  software  parts  have  been 
implemented  on  PDAs,  notebook  computers,  and  wearable  computers.  A  few  more  complex 
components  like  the  iGEN®  Cognition  Engine  Plug-in  tend  to  require  a  bit  more  computational 
power  than  available  on  current  PDAs,  although  it  has  been  employed  extensively  on  notebook 
computers  and  other  transportable  or  embeddable  computers.  The  intent  of  this  analysis  is  to 
determine  the  functionality  that  can  be  implemented  on  each  proposed  delivery  platform.  See 
the  Discussion  section  of  this  report  for  more  discussion  of  the  various  design  and  deployment 
tradeoffs  made  in  our  Phase  I  design  for  the  TAH-RI  sub-assembly. 

Understanding  the  Cognitive  Work  Needed  for  Battlefield  Decision  Making 

CHI  Systems,  and  the  proposed  Principal  Investigator  in  particular,  has  a  long  history  of 
successful  work  in  understanding  the  cognitive  work  needed  for  humans  or  intelligent  agents  to 
make  decisions  that  are  effective  and  timely.  The  COGNET  framework  has  been  hailed 
(Eggleston,  2004)  as  “...a  fully  developed  [Cognitive  Systems  Engineering]  framework...”  CHI 
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Systems’  prior  work  over  the  past  two  decades  has  resulted  in  development  of  a  framework  for 
representing  context  in  the  cognitive  terms  that  people  represent  and  maintain  it.  This 
framework  is  called  COGNET  (Zachary,  Ryder,  Ross,  &  Weiland,  1992),  and  incorporates  a 
notation  for  capturing  both  the  declarative  and  procedural  knowledge  of  a  work-domain 
ontology,  including  the  expertise  needed  to  generate  expert-level  behavior.  Later,  a  software 
engine  called  BATON  was  built  to  allow  COGNET  context  representations  to  be  authored  and 
directly  translated  into  software  (Zachary,  Le  Mentec,  and  Ryder,  1996).  BATON,  which  can  be 
thought  of  as  an  executable  cognition  engine,  is  one  of  several  components  forming  a 
commercially  available  integrated  toolkit,  called  iGEN®,  for  building  cognitive  model-based 
intelligent  agents. 

iGEN®  —  The  Cognitive  Agent  Software  Development  Toolkit 

TAH-RI  architecture  incorporates  the  iGEN®  Cognition  Engine  Plug-In  component  of 
C3Core,  based  on  COGNET  and  the  BATON  executable  cognitive  architecture  (see  Pew  & 
Mavor,  1998),  although  unlike  most  analogous  systems  it  was  created  for  engineering  purposes 
(i.e.,  as  a  vehicle  for  creating  practical  applications)  rather  than  as  a  platform  for  generating 
and/or  testing  psychological  theory.  Originally  created  as  an  engine  to  embed  user-models  into 
intelligent  interfaces  (Zachary,  Ryder,  Ross,  &  Weiland,  1 992),  the  system  has  been  generalized 
and  extended  over  time  to  create  a  flexible  framework  for  building  cognitive  agents  for  use  in 
intelligent  training,  decision-support,  and  human  performance  modeling  (see  Zachary,  Ryder, 
Santarelli,  &  Weiland,  2000).  iGEN®  is  an  integrated  software  development  environment  that 
supports  the  authoring,  editing,  debugging,  and  integrating  of  COGNET  models  (Zachary  & 
LeMentec,  1999).  Following  is  a  brief  summary  of  key  facts  about  iGEN®: 

•  Toolkit  for  building  intelligent  software  applications  /  interfaces 

o  Stored  expertise  model  representation  of  compiled  human  expertise  enabling 
performance  of  a  particular  job  /  role  -  not  a  scenario-specific  script 

■  Build  and  maintain  situation  awareness  (SA)  in  complex  environments 

■  Generate  appropriate,  competent  behavior  and  attention-shifts 

o  Integrated  Development  Environment  (IDE)  for  authoring,  debugging,  testing, 
and  reviewing  (e.g.,  W&A)  expertise  models 
o  Runtime  cognition  engine  based  on  human  cognitive  architecture,  executes 
expertise  model  to  generate  timely  expert-level  behavior 

•  Theory-neutral,  can  build  models  implementing  variety  of  different  cognitive  science 
theories  (e.g.,  for  memory;  processes) 

•  iGEN®  applications  in  situations  where  humans  unavailable 

o  Intelligent  &  distributed  simulation  (HLA  compatible;  synthetic  team-mates) 
o  Simulation-based  training  systems  (instructors/tutors  and  student  models) 
o  Decision/performance  support  systems  (embedded  advisors,  assistants) 
o  Automation  (intelligent  agents;  synthetic  co-pilot/engineer) 

•  Agent-based  Modeling  and  Behavior  Representation  (AMBR)  program  in  which 
USAF  (AFRL),  with  USN  (ONR),  evaluated  competing  architectures. 

This  patented  COTS  product  will  be  used  in  TAH-RI  (Figure  21)  to  provide  higher-level 
expertise  (1)  to  enable  off-board  or  on-board  mission  planning  for  robots,  and  (2)  to  augment 
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robots’  onboard  navigation  capabilities  through  integration  of  state  of  the  art  terrain  analysis 
capabilities  with  awareness  of  the  battlefield  situation  through  a  shared  CROP  and  means  of 
building  and  maintaining  battlefield  situation  awareness  for  planning  using  the  CROP,  (3)  to 
enable  rapid  deployment  of  reliable  Intelligent  Tutoring  Systems  for  training  human  Soldiers  in 
terrain  understanding,  and  (4)  to  enable  development  of  embedded  performance  support  systems 
to  help  human  Soldiers  reliably  operate  at  a  higher  level  of  expertise  in  integrating  terrain 
understanding  into  their  tactical  behavior. 


Figure  21.  iGEN®  Cognition  Engine  Plug-In  Overview 

COGNET  was  created  as  a  general  model  which  could  be  used  for  building  specific 
models  (or  model  instances)  of  human  cognition  and/or  behavior  for  a  range  of  specific  purposes 
(which  has  grown  over  time).  These  purposes  include  design/analysis  of  person-machine 
interfaces  and  human  work  tasks,  performance  diagnosis  in  intelligent  training  systems,  decision- 
support  and  performance-support  in  real-time  computer-based  work  environments,  and 
simulation  of  human  behavior  and  performance,  and  now  robot  planning  and  higher-level 
behavior  generation,  a  key  purpose  in  this  and  other  recent  projects.  As  new  COGNET  model- 
instances  have  been  built,  a  deliberate  effort  has  been  made  to  identify  features  that  are  common 
to  most/all  model  instances  and  to  retain  these  within  the  architecture,  while  maintaining 
customization  affordances  for  more-detailed  architectural  features  that  may  be  needed  (only)  for 
certain  types  of  model-instances.  This  process  has  been  motivated  by  theory,  ideally  in  the  sense 
noted  initially:  to  identify  useful  simplifying  assumptions  and  boundary  conditions  of  the  known 
assumptions. 

In  specific,  the  COGNET  cognitive  architecture  is  based  on  the  modified  stage  theory  of 
human  information  processing.  COGNET  incorporates  distinct  but  interdependent  declarative 
and  procedural  knowledge  representations.  On  the  procedural  side,  it  supports  hierarchical 
goal/subgoal/operation  procedural  knowledge  structures,  integration  of  cognitive  and  behavioral 
operations,  and  context-sensitive  modification  of  strategies  based  on  declarative  knowledge 
content.  On  the  declarative  side,  it  allows  the  notion  of  mental  model,  or  other  domain-specific 
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expert  constructs  to  be  incorporated.  Attention  management  is  an  emergent  property  of  the 
architecture  and  Principles  of  Operation,  supporting  multi-tasking  easily  and  naturally.  More 
recent  COGNET  developments  provide  metacognition  mechanisms  and  allow  a  model  to  be 
aware  of  its  own  processes  and  states.  This  cognitive  proprioception  allows  decision-making  and 
performance  to  be  modified  based  on  the  state  of  cognitive  processing.  These  mechanisms  were 
used  in  our  ATC  model  to  drive  task  load  based  strategies  and  error  generation. 

COGNET  provides  a  surprisingly  simple  core  architecture.  Representations  of  process  or 
structure  limitations,  such  as  memory  capacity,  decay,  or  recall  constraints,  visual  regions  and 
their  acuity  constraints,  etc.  are  not  included  in  the  core  architecture,  although  the  iGEN® 
implementation  is  extensible  and  allows  such  representations  to  be  added.  The  Principles  of 
Operation  that  describe  the  functionality  of  the  core  architecture  are  derived  from  macro-theories 
of  expertise,  including  Klein's  recognition  primed  decision  theory  and  Ericcson  and  Kintsch's 
long-term  working  memory  model.  As  noted  earlier,  both  here  and  in  other  applications 
COGNET/iGEN®  has  been  able  to  produce  highly  realistic  representations  of  human  behavior  in 
contexts  as  simple  as  the  AMBR  testbed  or  as  complex  as  that  of  advanced  Navy  command  and 
control.  While  some  might  suggest  that  the  lack  of  such  sub-process  models  makes 
COGNET/iGEN®  a  framework  in  search  of  a  theory,  the  authors  actually  see  the  results  as 
leading  to  a  quite  different  conclusion.  The  lack  of  embedded  theories  of  processing  limits  or 
other  sub-processes  makes  COGNET/iGEN®  a  simpler  and  more  parsimonious  model  than  other 
models  which  incorporate  these  components,  but  has  not  prevented  COGNET/iGEN®  from 
producing  high  quality  (and  in  many  AMBR  cases,  the  best- fitting)  behavioral  predictions. 

Thus,  it  can  be  concluded  that  for  the  purposes  of  human  behavioral  representation,  such 
architectural  features  are  not  necessary  and  do  not  pass  the  test  of  Occam's  razor. 

COGNET  does  place  substantial  emphasis  on  the  representation  of  knowledge,  both 
procedural  and  declarative.  It  provides  no  standard  or  atomic  level  of  representation,  allowing 
adaptation  to  the  modeling  goal,  but  the  representational  forms,  like  the  architectural  Principles 
of  Operation,  are  targeted  toward  the  forms  of  expert  knowledge  typically  used  in  generating 
skilled  performance  (rather  than  novice  performance). 

Distributed  Field  Robot  Architecture  (DFRA) 

Figure  22  provides  an  overview  of  how  autonomous  robots  make  navigation  decisions 
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Figure  23.  Nested  Hierarchical  Controller  Approach  to  Path  Planning  /  Navigation 


and  generate  behavior  sequences.  Figure  23  illustrates  how  the  nested  hierarchical  controller 
approach  to  path  planning  and  navigation  breaks  down  the  situation  into  different  scale 
problems,  with  the  highest  level  problem  description  up  at  the  Mission  Planner  level,  mid-level 
path  planning  and  consideration  of  navigational  constraints  considered  as  part  of  the  problem 
description  at  the  Navigator  level,  and  the  lowest-level  details  of  behavior  instantiation  and 
monitoring  of  progress  being  tackled  at  the  Pilot  level.  Figure  24  shows  a  simplified  overview  of 
the  new  TAH-RI  plug-in,  to  be  completed  early  in  Phase  II  development  using  the  Phase  I 
implementation  of  the  TRULLA  algorithm  onboard  the  designated  robotic  platform.  Figure  25 
illustrates  the  behavior-generation  process  from  a  robot's  perspective,  moving  from  sensed 
information  on  the  left  to  actions  taken  on  the  right.  Note  that  the  decision-making  needed  to 
generate  appropriate  behavior  at  the  actuator  level  is  divided  by  roboticists  into  two  different 
levels  of  behavior,  the  first  called  strategic  behavior",  but  not  tied  to  military  tactical  behavior 
deals  with  higher-level  actions  like  "follow  path",  while  the  second  level,  called  "tactical 
behavior"  by  roboticists  deals  with  the  lowest-level  component  actions  (e.g.,  "speed-control", 
"avoid",  "center-camera")  implemented  using  available  actuators  on  the  robot.  Figure  26 
illustrates  the  Distributed  Field  Robot  Architecture  which  integrates  these  elements  into  a  more 
general  robotic  architecture. illustrates.  Figure  26  illustrates  the  Distributed  Field  Robot 
Architecture  which  integrates  these  elements  into  a  more  general  robotic  architecture. 
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Figure  24.  Simplified  Overview  of  TAH-RI  Trulla  Plug-In  Deployed  On-Board  Robot 
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Figure  26.  Distributed  Field  Robot  Architecture  (DFRA) 

TAH-RI  Architecture 

The  architecture  for  the  TAH-RI  sub-assembly  to  be  implemented  in  Phase  II  is  basically 
a  combination  of  CHI  Systems  C3Core  components  with  our  iGEN®  cognition  engine  plug-in,  a 
set  of  tactical  behaviors  integrated  with  a  simulation  engine  as  the  Sim9  Plug-In,  and  at  least  one 
new  component  to  be  built  just  for  TAH-RI  (illustrated  in  Figure  24,  above)  and  one  existing 
component  (TERRAIN_V5/6)  that  will  be  significantly  adapted  for  TAH-RI  in  a  large  portion 
of  Phase  II  development  with  the  addition  of  a  large  number  of  significant  advances  in  storage, 
analysis,  and  spatial  query-based  retrieval  of  terrain  features  and  tactical/operational  elements  of 
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the  CROP.  Figure  20,  above,  also  illustrates  the  arrangement  of  these  and  other  potentially 
useful  components  in  the  TAH-RI  architecture.  Figure  27,  below,  illustrates  the  value  of  this 
composable  component-based  approach  (e.g.,  variety  of  delivery  configurations  possible)  to  the 
TAH-RI  architecture. 
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Figure  27.  Conceptual  Overview  of  Implications  of  TAH-RI  Composable  Architecture 

Task  8  Results:  Document  Progress  and  Report  Results. 

This  report  constitutes  the  draft  final  report  for  this  Phase  I  effort. 
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Discussion 


In  addition  to  the  specific  tasks  described  above,  this  Phase  I  effort  also  addressed  several 
overarching  issues  which  are  discussed  here.  We  start  with  the  TAH-RI  concept  of  operations 
(CONOPS),  followed  by  a  discussion  of  the  many  Human-Robot  Interaction  (HRI)  issues 
relevant  to  TAH-RI  design,  development,  and  employment. 

TAH-RI  Concept  of  Operations 

TAH-RI  is  intended  to  be  a  collection  of  software  components  integrated  to  work 
together  as  a  sub-assembly  within  some  larger  system.  For  instance,  TAH-RI  could  be  used  to 
help  rapidly  build  a  simulation-based  Intelligent  Tutoring  System  to  improve  readiness  and 
ensure  the  highest  level  of  competence  among  Soldiers  preparing  to  deploy,  who  need  to  be  able 
to  integrate  and  apply  terrain  understanding  with  intelligence  information  and  their  CROP  to 
guide  their  tactical  behavior  in  situations  where  the  terrain  is  complex  and  urban.  In  this  case, 
TAH-RI  already  includes  the  iGEN®  cognition  engine,  which  has  proven  its  value  in  training 
systems  (Glenn  et  al,  2003;  Zachary,  Ryder,  and  Hicinbothom  1998;  Zachary,  et  al,  1998; 
Zachary,  Ryder,  Hicinbothom,  2000),  providing  means  of  rapidly  constructing  synthetic  team 
members  (SYNTHERs)  and  synthetic  instructors  which  call  collaborate  to  assess  student 
knowledge  state  in  simulation-based  Intelligent  Tutoring  Systems.  The  beginnings  of  an  ITS 
could  be  constructed  by  combining  TAH-RI  with  an  appropriate  Crew  Station  and  underlying 
simulation  (e.g.,  through  OTB  or  other  HLA/DIS  external  simulations,  or  even  through  the  SIM9 
component  providing  smaller-scale  simulation  capabilities  appropriate  for  many  ITS 
applications.  The  heart  of  the  ITS,  its  assessment,  record-keeping,  and  instructional  intervention 
expertise  could  come  from  one  of  many  Learning  Management  Systems  and  ITS  development 
efforts  within  DoD,  including  DARPA’s  DARWARS  system  of  systems  for  simulation-based 
training. 

For  robotics  applications,  TAH-RI  could  be  used  on  a  variety  of  robots,  in  several 
configurations,  as  illustrated  in  Figure  28,  below.  In  this  application,  TAH-RI  is  utilized  in  three 
distinct  configurations: 

•  First,  as  an  integrated  on-board  cognitive  agent  system  ,  CA(SW)a  integrated  with 
Robot  A  (HW/SW)  through  the  new  TAH-RI  component  illustrated  in  Figure  24. 

•  Second,  as  a  stand-alone  off-board  cognitive  agent  system,  CA(SW)b  aboard  a 
command  vehicle  or  other  C2  node.  Note  that  in  this  configuration  and  the  first,  the 
TAH-RI  cognitive  agent  would  also  listen  to  orders  and  information  from  the 
commander  to  the  robot(s)  so  that,  like  good  team  members,  they  overhear  essential 
information  and  help  keep  their  situation  awareness  (SA)  about  their  robots’  status 
and  plans  and  missions. 

•  Thirdly,  as  a  small  embedded  component  aboard  Robot  B  (HW/SW)  consisting 
primarily  of  the  new  TAH-RI  component  illustrated  in  Figure  24  along  with  the 
C3AF  core  and  “typical”  components  in  a  sub-assembly  needed  to  communicate  with 
the  distant  commander,  CO(Hum)  and  with  CA(SW)b  TAH-RI  sub-assembly. 
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Figure  28.  TAH-RI  CONOPS  for  Autonomous  Robotic  Vehicles 

Human-Robot  Interaction  with  TAH-RI 

Human  robot  interaction  has  traditionally  addressed  technical  advances  in  robotics 
allowing  for  communication  between  human  and  robotic  components  of  a  system.  With 
technological  advances  in  communication,  a  recent  shift  to  more  human-centered  HRI  is 
underway.  A  series  of  workshops  sponsored  by  DOE,  the  National  Science  Foundation,  and 
Defense  Advanced  Research  Project  Agency  (DARPA)  has  provided  the  impetus  for  much 
human-centered  HRI  work  (DOE,  1998a;  Murphy  and  Rogers,  2001). 

HRI  in  the  current  context  involves  the  active  acquisition  of  information  from  robots,  the 
transformation  of  that  information  into  displays  that  help  humans  construct  situation  awareness, 
as  well  as  into  representations  upon  which  the  robot  can  similarly  reason,  and  the  network-based 
propagation  of  critical  information  in  real  time  under  communication  failures  and  congestion. 
Figure  29  illustrates  Human  Robot  interaction  factors  relevant  to  humans  interacting  with  TAH- 
RI.  Three  dimensions  are  identified:  the  role  of  the  human  in  the  human-robot  system,  team 
design  issues  including  the  human  to  robot  ratio  and  the  proximity  of  team  members,  and  levels 
of  autonomy  (Bruemmer,  Marble,  Dudenhoeffer,  Anderson,  &  McKay,  2002). 
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Figure  29.  HRI  considerations  with  TAH-RI 

Human  Role 

Scholtz  (2003)  describes  four  human  roles  in  human-robot  osinteracting  : 
ConsumerOperator,  Bystander,  and  Teammate.  Warfighters  on  the  battlefield  may  find 
themselves  in  any  one  of  these  roles  in  interactions  with  TAH-RI  :teammate. 

•  Consumer  -  Terrain  data  obtained  from  the  robot  will  be  utilized  by  humans  (and  other 
robots  and  intelligent  agents  and  incorporated  into  mission  planning  and  decision  making 
processes.  HRI  must  assess  user  needs  and  address  graphical  user  interface  requirements  that 
support  the  understanding  of  commander’s  intent,  and  development  of  situation  awareness 
(Endsley),  while  mitigating  against  the  propensity  toward  cognitive  overload.  Identifying  user 
requirements  and  supporting  the  interpretation  of  commander’s  intent  and  SA  development 
during  robot  assisted  task  performance  is  a  critical  to  effective  human  robot  interaction  with 
TAH-RI  Operator  -  exhibiting  varying  degrees  of  human  control  over  the  robot  based  on  the 
robot’s  level  of  autonomy.  The  human  robot  operator  will  always  be  in  the  loop;  even 
autonomous  robots  will  require  significant  human  oversight  and  frequent  intervention.. 

•  Bystander-  humans  and  robots  tasks  are  independent  of  each  other  -  the  human  may  have 
little  or  no  knowledge  of  the  robot.  Much  of  the  existing  human-robot  interaction  research  has 
addressed  the  goals  of  social  acceptance  of  robots  or  interface  design,  notably  Breazeal,  2000; 
Arkin,  Fujita,  Takagi  and  Hasegawa,  2003;  Draper,  Pin,  Rowe  &  Jansen,  1999;  Wilkes,  Alford, 
Cambron,  Rogers,  Peters  &  Kawamura,  1999;  Khatib,  Yokoi,  Brock,  Chang  &  Casal,  1999. 
Research  suggests  that  people  perceive  autonomous  robots  differently  than  they  do  most  other 
computer  technologies.  HRI  with  TAH-RI  during  terrain  analysis  in  the  field  will  necessarily 
require  consideration  of  social  behaviors;  that  is,  human  responses  to  and  ‘interpersonal’ 
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interaction  with,  robots  (e.g.,  issues  trust,  emotion,  and  attributions  of  being  human-like). 
Implications  of  bystander  interactions  includes  a  requirement  that  the  robot  be  able  to 
communicate  and  explain  it’s  goals,  plans  and  actions  to  nearby  humans. 

•  Teammate  -  humans  and  robots  working  collaboratively  on  tasks  to  accomplish  shared  goals. 
HRI  with  TAH-RI  must  be  supported  by  providing  both  the  human  and  the  robot  with 
information  in  usable  formats.  For  the  robot  this  means  that  information  must  be  represented 
in  a  format  that  the  robot  can  recognize,  ‘understand’  and  use.  Given  specific  mission 
contexts,  HRI  becomes  increasingly  important,  TAH-RI  offers  the  robot  within  the  human  - 
robot  system  the  ability  to  understand  and  reason  about  terrain,  and  to  anticipate  or  tailor 
interactions  with  the  human  based  on  mission  goals.  In  addition,  especially  in  field 
applications,  the  robot  will  need  assistance  from  the  human  to  resolve  uncertainties  and  to 
accomplish  tasks.  TAH-RI  identifies  underlying  terrain  features  and  offers  some  level  of 
reasoning  about  those  features.  TAH-RI  also  provides  common  terrain  referents  to  anchor 
human-robot  interactions. 

For  example,  UGVs  must  be  able  to  navigate  through  the  battlespace.  Not  only  will 
robots  help  gather  terrain  data  for  interpretation  and  use  by  human  team  members,  but  the  robot 
must  also  be  able  to  recognize  and  interpret  that  terrain  information  available  in  terrain 
databases.  Data  providing  shared  referents  for  the  development  of  ‘common  ground’  are  critical 
for  coordinating  human  team  activities  (Kraut,  Fussell,  Brennan,  &  Siegel ;  Olson  &  Olson, 
2000).  Human-robot  teams  will  also  be  tasked  with  developing  common  ground  -  a  common 
understanding  of  the  situation  based  on  objects  of  referents  -  in  order  to  coordinate  activities. 

(For  example,  once  a  terrain  feature  is  identified,  the  human  and  robot  can  collectively  reason 
about  the  implications  of  that  feature  for  mission  accomplishment. 

Proximity 

Autonomous  robots  are  a  distinctive  case  in  HCI  in  that  robots  are  mobile;  resulting  in 
differing  levels  of  physical  proximity  with  other  robots,  people,  and  objects.  Proximity  has  an 
impact  on  the  expected  form  of  human  robot  interaction.  Mobile  robots  will  have  to  negotiate 
their  interactions  in  a  dynamic,  sometimes  physically  challenging,  environment  (e.g.  Burke, 
Murphy,  Coovert,  &  Riddle,  2004;  Yanco,  Drury,  &  Scholtz,  2004).  For  example,  humans  may 
be  interacting  with  the  robot  but  be  distributed  from  the  robot  (e.g.,  a  human  in  a  command  post 
interacting  with  a  robot  in  the  field)  and  humans  may  also  be  co-located  with  the  robot  (e.g., 
Soldier  walking  along  side  a  robot). 

■  Distributed  -  human-robot  system,  human  robot  interaction  will  likely  focus  on  issues  of 
human  supervising,  monitoring  and  controlling  the  robot,  and  issues  related  to  acquiring, 
interpreting  and  integrating  the  terrain  information  obtained  by  the  robot  in  order  to 
develop  appropriate  situation  awareness. 

■  Co-located  -  humans  may  be  side  by  side  with  robots,  engaged  in  independent  tasks 
(e.g.,  reconnaissance)  or  they  may  be  working  collaboratively  to  accomplish  a  common 
goal.  In  the  first  case,  working  side  by  side  with  a  robot,  it  is  important  to  consider  that 
people’s  perceptions  of  autonomous  robots  are  often  more  anthropomorphic  than  their 
perceptions  of  other  systems  (Friedman,  Kahn,  &  Hagman,  2003).  In  the  second 
instance,  working  collaboratively  on  common  tasks  requires  an  understanding  of  team 
behaviors. 
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Human-Robot  Team  Configuration 

In  addition  to  proximity,  specific  team  configuration  also  has  implications  for  interacting 
with  TAH-RI.  Yanco  &  Drury  (2002)  describe  a  number  of  human-  robot  configurations  based 
on  the  intersection  of  two  dimensions  -  the  ratio  of  people  to  humans  and  the  level  of  shared 
interaction  among  teams:  1)  one  human  controlling  one  robot,  2)  one  human  controlling  a  group 
of  robots,  3)  one  human  controlling  multiple  independent  robots  (robots  not  coordinating 
amongst  themselves),  4)  a  human  team  collaboratively  issuing  one  command  to  one  robot,  5) 
multiple  humans  controlling  one  robot,  6)  a  team  of  humans  controlling  a  team  of  robots 
(collaborative),  7)  teams  of  humans  controlling  multiple  independent  robots,  and  8)  multiple 
humans  issuing  commands  to  a  robot  team. 

Currently,  much  of  military  HRI  research  is  focused  on  single  human  operator 
supervision  of  either  small  groups  of  loosely  coupled  robots,  or  large  groups  of  tightly  coupled, 
autonomous  robots  (Young,  Emmerman  &  Nguyen,  2002).  As  described  in  the  introduction  of 
this  report,  various  attempts  at  controlling  multiple  robots  in  the  field,  requiring  individualized 
user  intervention,  have  failed. 

Level  of  Autonomy 

Across  the  battlefield  robots  exhibiting  differing  levels  of  autonomy  and  requiring 
varying  levels  of  control.  Bruemmer  and  Waldon’s  levels  of  autonomy  support  the  various  roles 
of  the  human  in  the  human-robot  team: 

•  Fully  autonomous  -  the  robot  plans  its  own  path,  and  responds  to  the  environment  requiring 
no  input  from  the  user  except  for  high-level  tasking. 

•  Shared  control  -  robot  has  the  initiative  to  plan  its  path  and  respond  to  the  local  environment. 
Robot  queries  human  user  for  information  and  problem  solutions. 

•  Safe  Operations  -  the  robot  is  controlled  by  the  human  user,  however  the  robot  can  take 
safety  initiatives.  For  example,  in  safe  mode,  if  the  robot  senses  an  obstacle,  and  the  operator 
navigates  into  that  obstacle,  the  robot  decides  the  command  is  not  safe  and  will  stop  its 
movement  to  avoid  collision. 

•  Teleoperation  -  the  robot  is  continuously  controlled,  at  a  low  level,  by  the  human  operator 
(except  when  communications  drop  out). 

Mixed  initiative  HRI  with  TAH-RI  requires  robots  and  humans  are  able  to  ‘understand’ 
and  predict  each  other’s  behavior  (i.e.,  the  robot  has  a  model  of  human  behavior  and  the  human 
has  a  model  of  robot  behavior).  In  interacting  with  TAH-HI,  the  system  must  also  be  able  to 
recognize  conditions  in  which  human  intervention  is  required,  and  when  robot  intervention  is 
required  and  be  capable  of  accepting  variable  levels  and  frequencies  of  intervention.  For 
example,  during  path  planning  the  commander  may  have  directed  the  robot  to  follow  roads  or  go 
through  a  particular  checkpoint,  but  the  main  road  may  be  blocked,  and  instead  it  needs  to  make 
up  time  by  taking  a  short  cut  on  a  secondary  road  or  trail.  This  requires  it  to  relax  a  constraint, 
which  generally  requires  permission  from  a  higher  cognitive  agent  (usually,  the  human). 
Additionally,  robot  must  be  able  to  explain  why  it  is  doing  what  it  is  doing  to  the  human. 
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